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1 INTRODUCTION 


1.1 Project Goals 

For-ythe past two and a half years, we have performed a study 
of the feasibility o,f the use of Charge Transfer Devices (CTDs) in 
the classification of multi-spectral image data. This work consis- 
ted of two primary stages: 1) An evaluation of particular’ devices 

to determine their suitability in a matrix multiplication sub- 
system of a pattern classifier and 2) The design of a’ prototype of 
such a system if a suitable device was found. The work centered 
around "analog -analog correlator" devices which consist of two 
tapped delay lines , on chip multipliers , and a summed output ; these 
devices will be discussed in more detail in Section 2 of this report. 

1.2 Summary of Results 

In general, our results have been encouraging; one of the 
tested devices showed performance characteristics which warranted 
further development, the design of the system was accomplished, and 
construction was begun. Reference is made to two previous progress 
reports dated December 20, 1977 and August 28, 1979. These papers 
contain detailed reports on the results to the time of their issue, 
and the material contained in them will only be summarized in this 
report . 

The previous reports indicated the following findings : 


(1) The first device evaluated, the Reticon AAC-32 

was found to be not suitable because of serious linearity problems. 

(2) A second series of devices, the Reticon R5402/5403, 
was tested and was found to have acceptable accuracy and reason^ 
able linearity indicating a need to explore its use in a pattern 
classifier. 

(3) An architectural design and part of the detailed 
design for a multispectral classifier using the Reticon devices 
and controlled by an LSI-11 microcomputer was completed. 

(4) Software was developed to support communication 
between the LSI-11 and a VAX medium scale computer, This software 
allowed the reading of LANDSAT tapes and the subsequent transfer 
of the multispectral images to the LSI-11 floppy disk for use by 
the classifier. 

(5) Software was also developed to support the display 
of false colored images on a Chromatics graphics terminal. 

Additional accomplishments since the August report are: 

(1) The design for the classifier was completed. 

(2) A printed circuit layout for the analog boards was 
completed, and the boards were fabricated. The other boards will 
be wire -wrapped . 

(3) A test jig for the analog board was built and check- 
out begun. 

(4) System software development was begun. 


2 CTD THEORY 


This section contains a brief overview of Charge Transfer 

Device characteristics. For a detailed explanation of CTD 

construction and operation see the 1977 report and its references, 
v 

A CTD is simply a monolithic integrated circuit which moves 
packets of charge linearly in synchronism with a clock. Depending 
upon the application,, these charge packets may be used to repre- 
sent either digital or analog information. CTDs have been, used for 
a wide variety of purposes, particularly analog signal processing, 
digital memories , and imaging arrays . 

Two very common types of CTDs are the "Bucket Brigade 
Device" (BBD) and the "Charge Coupled Device" (CCD) . The major 
difference between these two devices is the manner in which the 
charge packets are stored and transferred from cell to cell. 

This difference results in slightly differing performance char- 
acteristics between the two types . 

A common use of CTDs is signal processing has been as simple 
analog delay lines or shift registers. By modifying the clocking 
electrodes complex functions of the input rather than simple 
delays have been realized. These functions are of the form: 

E a^b^ where the b^s are samples of the input signal and the a^s 
are weighting coefficients. Such devices are known as "fixed 
tap weight devices" (from their filtering applications) because 
the weights are determined at manufacture and cannot be changed 
afterward. Fixed tap weight devices have been used to produce 
a variety of functions such as matched filtering, correlation, 
or the magnitude of the discrete Fourier transform. 


, The versatility of the CTD may be extended by using two 
delay lines , adding analog multipliers at each point, and 
providing a mechanism for summing the outputs of the multipliers . 
Such a device may be used to perform sum-of -product operations 
in which both of the operands are arbitrary. Experimental samples 
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of these "variable tap weight devices" are now available, and 
these devices are the- subject of this study. 


3 PATTERN CLASSIFICATION USING CTDS 


A major bottleneck in pattern classification operations has 
been the matrix multiplication required in the calculation of the 
discriminate function; 

Ix-pT^J T lx - p A ] 

k» 

where x is the data point vector, u is the mean vector of the ith 
class, and C^ (also commonly signified by E^) is the covariance 
matrix of the ith class . 

Figure 3 . 1 illustrates a proposed layout of sum of product 

CTDs (SOPs) which may be used^to calculate the discriminant 

function. In this arrangement each of the first group of SOP 

devices (E^ --E ) perform one of the row-column multiplications 
_ *r _1 

of the x-v^ C^ term of the discriminant function calculation. 
The results from these operations are then multiplexed into the 
final SOP device along with the x-fi\ term producing the desired 
final result . It is seen that this configuration requires at 
least two CTD loading times to perform the calculation. 

A reduction of the calculation time to one SOP device loading 
time would be highly desirable. Figure 3.2 shows an arrangement 
which accomplishes this reduction by eliminating the multiplexer 
and the final CTD. The diagonal symmetry of the covariance matrix 
allows this simplification because the matrix may be transformed 
into upper triangular form. In this case the SOP devices perform 
the x — ik calculation as before. These results are then each 

squared and the results summed to produce the answer . 


























4 DEVICE EVALUATIONS 


4.1 AAC-32 

V 

Once the matrix multiplier configuration was derived an 

# 

evaluation of the available devices was needed. A detailed 

account of the evaluation techniques and testing set-up hard- 

* 

ware is given in the 1977 report . , 

The first device evaluated was the Reticon AAC-32 , a bucket 

* 

brigade device. This chip contains two tapped 32 cell delay lines 

with an untapped or "dead" cell at the start of one delay line 

* 

and the end of the other. Thirty-two on-board multipliers and a 

« 

summer compute the sum-of-products function. 

After determining values for the various bias voltages and 
currents required by the chip, tests were made to determine proper 
signal levels and to evaluate the performance of the device . 

Zeroing and signal level adjustments were satisfactorily executed, 
then linearity, accuracy, and repeatability were examined. 

The results showed an accuracy of about 5 bits and a repeat- 
ability of about 8-9 bits with the variance appearing to be the 
result of random noise. Also noted were short«term and long-term 
drifts when the device was left unclocked and a start-up inaccuracy. 
These results all appeared to indicate that the device was accept- 
able for a matrix multiplication application. 

The AAC-32 however, showed a exhibited a serious linearity 
problem; in one quadrant, the output was severely clipped. 
Adjustments and reductions of the input could not remove the 
clipping. The failure appeared to be caused by the multipliers; 
observations of the delayed signal after it passed through the 


bucket brigade showed no distortion problems. This* non-linearity 
caused the AAC-32 to be ruled unsuitable. 

4.2 R5402/5403 Series 

V 

The Reticon R5402/5403 chips are revised versions of the AAC-32 
which do not exhibit many of the earlier chip's problems. The 
difference between the 5402/5403 chips is in the number of* taps: 
the R5402 has 16 taps; the R5403 has 32. A major change in both 
chips from the AAC-32 is the presence of a string of storage 
capacitors on one side of the device. The input signal on that 
side passes down the string of capacitors and is passed in parallel 
to the bucket brigade upon a sample and hold strobe signal. 

The tests performed on the AAC-32 were repeated on the 
R5402/5403, and the results were encouraging. The chips exhibi- 
ted acceptable accuracy (^7-8 bits) and repeatability and did 
not have the linearity problems of the AAC-32. The conclusion 
of the evaluation of the R5402/5403 was that the chips’ perfor- 
mance was marginally satisfactory and that the design of a classi- 
fier around these devices should proceed. 

5 CLASSIFIER SYSTEM DEVELOPMENT 
5 .1 Organization 

Figures 5.1 and 5,2 illustrate the functional blocks and 
data flow of the classifier system. Figure 5.1 shows the set-up 
of a general classifier system while Figure 5.2 shows the set-up 
of the prototype system designed around the Reticon devices. 
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Figure 5.1 Block Diagram of 

General Classification System 






In Figure 5.1 the sensor input could be directly from a 
LANDSAT sensor after optical, and geometric corrections have been 
made, and the dat i source could be RAM built into the system for 
storing the necessary mean and covariance information. In the 
prototype system, however, the LSI-11 is the controlling micro- 
computer while the floppy disk serves as the sensor input by 
storing a LANDSAT image. All of the classifier hardware: CTD's, 

I* 

ft 

signal conditioning and conversion circuitry, RAM, and controlling 

I 

circuitry is contained in one block, and communication with the 
LSI-11 is over two parallel data buses (16 bits in each direction 
per interface) . The terminal is a 512x256 point color graphics 
display and serves as an output for the classified images . A 
more detailed description of the system is given in the 1979 report. 


5 . 2 Hardware 


5.2.1 Classifier Module Organization 

Thi| section contains a description of the classifier modules 
and their operation, These modules are: analog boards, timing 

and control/interface, and analog-to-digital conversion. Figure 
5.3 indicates how these modules are integrated to form the class- 
ifier system. The modules all plug into a backplane over which 
the digital TTL level signals and power pass, The analog signals 
are not sent over the backplane over which the digital TTL level 
signals and power pass. The .analog signals are not sent over 
the backplane to reduce potential noise problems; instead, they 
are passed from board to board by shielded cables. Except fo 
the A/D converter data output, a single board provides the required 
interface with the LSI-11. 

5.2.2 Analog Board 

The major functional component of the classifier is the analog 
or sum-of-products board. Eight of these boards form the core 
of the classifier; each contains a CTD and squaring circuit; the 
necessary D/A conversion, signal conditioning, and bias circuitry, 
and the RAM which is loaded with the appropriate column of the 
covariance matrix inverse. Upon receipt of the proper data and 
control signals each of these boards will cycle the CTD producing 
the row-column vector dot product and squaring the result. An 
analog multiplexer is included for diagnostic monitoring of criti- 
cal on-board signal points. 



Figure 5.3 Classifier Module Organization and Data Flow 






Appendix A contains the schematic of the analog board along 
with a diagram of the printed circuit card layout for the board 
and a parts list. 

5.2.3 Timing and Interface Mo dule. s 

The functions of the timing and control module and the inter- 

»» 

face module are closely intertwined; for that reason, they ’will 
be discussed together in this section. Clocking, control, data 
steering, and interfacing with the LSI-11 are accomplished by 
these modules. Ten circuit groups make up the two modules: 

(1) Transition Detector (TD) 

(2) Counter Clock Reset/Enabler (CDREC) 

(3) Counter and Decoder Circuit (CDC) 

(4) Clock Generator Circuit (CGC) 

(5) Address Bus Arbitrator (ABA) 

(6) Mode Decode (MD) 

(7) Sub tractor (SUB) 

(8) A/D Conversion Decoder (ADCD) 

(9) Master Clock Generator (MCG) 

(10) Chip Enable Decoder (CED) 

The instructions arrive from the LSI-11 in the form of a 
16-bit control word (see Figure 5.4). There are three basic 
operating modes for the classifier: LOAD, DIAGNOSE, and RUN. 

In LOAD mode the LSI-11 has control of the address bus and is load- 
ing new information for the classifier to process. In the DIAG- 
NOSE mode the LSI-11 is driving different parts of the classifier 
to test their performance. In both of these modes the LSI-11 
controls the backplane bus, and the CTD output data is invalid. 



In the RUN mode classifier operations proceed normally with the 
backplane bus controlled by the timing circuitry, These modes 
are selected by the mode selection bits (13-14) of the control word 
which are decoded by the Mode Decoder. 

The Address Bus Arbitrator is a three-line multiplexer that 
has, as ilnputs, the three low order address bits of the control 
work and the three low order bits from the CDC, and has the 
three low order bits of the backplane address bus as outputs . 

When the classifier is in LOAD or DIAGNOSE mode the Arbitrator al- 
lows the LSI-11 to control these three bits, otherwise the timing 
circuitry exercises control. 

The RAM selection bits (3,4,5 of the control word) go to 
the Chip Enable Decoder, This circuit has 6 output lines; the 

I 

RAMs on the analog boards are enabled, two at a time, by four of 
these, the x-RAM is enabled by the fifth, and the p-RAM is enabled 
by the last. During RUN mode this circuit is bypassed and all 
RAMs are enabled. 

The i)ata Output Bus Enable selects the pair of RAM output 
bus drivers to be enabled. This selection is necessary to avoid 
having two RAMs attempting to control the backplane bus . This 
circuit is located on each of the analog boards . 

The Subtractor is simply an ALU set to continually perform 
the necessary subtraction to obtain the x-p" term. The inputs come 
from the x-RAM which is loaded once per pixel and the p-RAM whir* 
is loaded once per image. 

The Transition Detector generates a single pulse to initiate 
a new cycle of the CTDs to produce the discriminate function 
results. This pulse is generated when the classifier is in the 
RUN mode and a change is detected on the three high order address 
bits . 


The Counter Clock Reset/Enabler generates the signals to 
clock the CTD. When the CDC receives a Counter Enable signal 
from the CDREC It will begin the loading of the CTD. The Clock 
Enable signal Is sent to the CGC to enable the CTD clocks. 

In addition to the Counter Enable signal, the CDC receives 

the Master Clock from the MCG. The data Is loaded Into the CTD 

* 

twice* (assuming an eight-feature vector) ; this method should 
yield better accuracy' than loading once and padding with zeros. 
During loading four outputs are produced. Three of these are low- 
order addresses and go to the address arbitrator. The other out- 
put Is the Count Finished signal issued when the loading cycle 
is complete. 

In addition to t^he Clock Enable and Count Finished signals 
the CGC receives the Master Clock signal and the three decoded 
mode signals . The clocks are always enabled during the LOAD 
and DIAGNOSE modes , and during RUN mode the clocks are started 
when the Count Enable signal is true . When the Count Finished 
signal is received one more clock cycle is applied to the A side 
of the CTD to correctly align the data because of the ’’dead" cell. 
The clocks are now halted, and the CTD is strobed to pass the 
data from the capacitors to the bucket brigades . The Load Ready 
strobe is now sent to the A/D Conversion Decoder. 

The A/D Conversion Decoder generates the convert command that 
activates the A/D converter. In Run mode this command is con- 
trolled by the Load Ready Strobe, and in the other modes it is 
under the control of the LSI-11. The signal "A/D status" is 
generated by the A/D converter upon completion of its operation. 
This signal is used to restart the CTD clocks to avoid drifts and 
to signal the LSI- 11 to read the A/D. 


5.2.4 A/D Board 


The A/D converter is the same 12 bit, 30psec Analog Devices 
module used in the chip evaluations and operates in the same 
fashion. v 

In addition to the A/D converter the board contains the 
summers for the eight signals from the analog boards. Switching 
is also provided between the summer output and a diagnostic 
signal jack used for monitoring various points on the analog 
boards during testing . 

5.3 Testing Software 

« 

Special arrangements were necessary for testing of the ana- 
log boards . A testing stand was constructed to allow the board 
to be exercised without the full classifier. A subset of the 
control word signals which was necessary for the operation of 
the analog board was used. Clocks, control, and data which are 
normally provided internally were added. The revised communi- 
cation word formats are shown in Figure 5.5. The purposes of the 
system were to check out the RAM on the board and to adjust the 
signal levels and offsets of the CTD inputs . 

The adjustment algorithms are similar to those used for the 
device evaluation. These algorithms are discussed in the previous 
reports. We are including two new RAM testing programs here; 
Appendix D contains program listings . 
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1. Program RAMTSB 


RANT SB checks the RAM in a bit-by-bit fashion. It loads 
the RAM with 1 at the first bit of the first location then checks 
the output * It then loads the RAM with 0 and checks the output . 

If the outputs are not consistent with the input data an error is 
logged. The testing sequence is from the first bit of the first 
location to the first' bit of the last location (63) . The program 
then checks the second and remaining bits in the same manner. 

* 

2. Program RAMTST 

*• 

This program is »an abbreviated version of RAMTSB which tests 
the RAM in a byte-by-byte fashion. It will not find all possible 
errors, but it may serve as a quick first test of the RAM opera- 
tion. The program operates by loading each location with all l's 
and all 0's and examining the result, 

6 CONCLUDING REMARKS 

The principal goal of this project has been the evaluation of 
Charge Transfer Devices and their potential use in pattern classi- 
fiers. As such, much of the work has been centered around testing 
the devices rather than construction of a final system. Since 
the devices were experimental and the application was new the 
work often involved a "ground-up" approach. In the case of the 
AAC-32, for example, very little information was available regard- 
ing optimum operating points, etc, In addition, some traditional 
types of tests were found to produce misleading results, for 


example, the correlation of two sampled analog sine waves appeared 
to produce very good performance i.e., a high siqnal- 
to-noise ratio. It was not until the testing methodo- 
logy was rethought and a different form of test was applied that 
the problems with the device became evident . 

t 

Three general conclusions may be drawn from the work: 

(1) With the advent of "variable tap weight" devices there 

I 

is a strong potential for the use of CTDs in pattern classifiers . 
They can be used to provide matrix multiplication subsystems. 
Suitable architectures were developed in this project, and ‘their 
potential . performance i3 good. The time required to produce an 
answer has been reduced to one CTD loading time. Further, such a 
classifier architecture allows the parallel computation of all of 
the discriminate functions (i.e. up to eight in this case) at 
once. Such a system of very fast, low power classifiers could be 
of tremendous benefit in processing data from sources such as 
LANDSAT satellites, particularly by making on-board classification 
feasible. This could make the use of pattern classification 
techniques and satellite data much more widespread, and could 
open the door to new uses of the techniques which are not possible 
now because of prohibitive computational requirements and their 
resulting delays. or high cost. 

(2) The evaluation of the Reticon AAC-32 clearly showed 
that it is not suitable for use in a classifier application. The 
failure did not, however, rule out the use of CTDs in classi- 
fiers because it appeared to result from a design problem in the 
analog multipliers rather than the CTD technology. It is inter- 
esting to note that if the failing quadrant was avoided, the AAC-32 
did show usefulness in other applications such as programmable 
transversal filters. (See Appendix C) . 


The Reticon R5402/5403 indicated the validity of the conclu- 
sions since it did not have the problems of the AAC-32. The 
performance of these new devices Indicated that they were candi- 
dates for matrix multiplication applications . 

(3) A design for a micro -computer controlled classifier 
using the RS402/5403 chips was accomplished, and the paper system 
has a reasonable size, complexity, and power consumption. The 
further devleopment of the system for testing and evaluation pur- 
poses is recommended. 

This project has by no means concluded that a CTD based 
classifier will operate well, only that such a system appears 
to be possible. The only was to accurately guage the performance 
of a CTD based classifier is to build and test it as is recommended 
Several questions remain which cannot be answered simply by testing 
devices. These questions include: the stability of the system, 

particularly the analog interface and signal conditioning circuit- 
ry; the effects of long term drifts and aging on the accuracy of 
the total system when added together; the speed at which the 
total system may be operated, and the reliability of the devices 
when operated in such a system. The CTD technology is developing 
rapidly, and performance characteristics can certainly be expected 
to improve in the future. This project and other novel appli- 
cations of the devices will certainly aid in indicating areas of 
technology development which need improvement and would increase 
the chances of the production of future devices which could 
operate even better. 


APPENDIX A 


ANALOG BOARD SCHEMATIC AND PARTS LIST 
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The schematic is oversized and has been sent to NASA under 
separate cover. 

y 
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Analog Board Pin Definitions 


Edge View from Pins 



All signals TTL compatible x Depends upon slot 

unless otherwise specified. 

* This signal is 0-+15V 

All unnamed pins reserved 
for future expansion. 


Computer Side 


I 

4 — 


Timing Board Pin Definitions 

* 

Edge View from Parts 



I All signals TTL compatible 

unless otherwise specified. *This signal is 0-+15V 


All unnamed pins reserved 
' for future expansion. 
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A/D Board Pin Definitions 





All signals TTL compatible 
unless otherwise specified. 


All unnamed pins reserved 
for future expansion. 



LAYUUJ UF T Ht£ ANALLC BumI'L* 


The board used is a double sided PCD* with 72 possible gold finger contact 
evenly distributed in both sides* the space between contacts is . 125 in. * the 
base material used is FR4# the overall size of the board is , 

9. 1 x 5. 2 in, 

PARTS LIST 

RESISTANCES 


The following values are given in Kohms and are assumed to be of 5% 
tolerance ani^ 1/4 W maximum dissipation* unless otherwise especified. 


location 

value (Kohms ) 


location 

value (Kohms ) 

R1 

1. 


R41 

150 

R2 

10, 


R42 

62. 

R3 

20, ’’ 


R43 

62, 

R4 

1 . 


R44 

100. 

R5 

10. 


R45 

15. 

R6 

20, 


R46 

2. 

R7 

10. 


R47 

68. 

RS 

1 . 


R48 

300. 

R9 

1. 0 


R49 

33. 

RIO 

10, 


R50 

100. 

Rll 

20. 


R51 

100. 

R12 

1 . 


R152 

1 . 

R13 

20. , 


R53 

100. 

R14 

20. 


R54 

1 . 

R15 

2. 


R55 

10. 

R16 

10. 


R56 

5. 1 

R 17 

20. 


R57 

1. 

R18 

2, 


R58 

10. 

R19 

10. 


R59 

5. 1 

R20 

20. 


R60 

1 . 

R21 

10. 


R61 

. 62 

R22 

• 4. 7 


R62 

. 62 

R23 

20. 


R63 

, 62 

R24 

10. 


R64 

. 62 

R25 

4. 7 


R65 

. 62 

R26 

10. 


R66 

, 62 




R67 

. 62 

R28 

20. 


R68 

. 62 

R29 

1 . 


R69 

. 62 

R30 

56. 


R70 

2. 

R31 

15. 


R71 

2. 

R32 

100, ohms 


72 

2. 

R33 

20. 


R73 

2. 

R34 

270. 


R74 

5. lohms 

R35 

100. 


R75 

5. lohms 

R36 

62, 


R76 

. 47 

R37 

160. 


R77 

. 47 

R3S 

120 


R78 

. 47 

R39 

150. 


R79 

. 47 

R40 

100. 


R80 • 

20. 


CAPACITORS 

The following capacitors are given in uF with 15 Vdc ratings and 
tolerances better to 10% unless otherwise specified. 


location 


Vs. lue 


value location 


Cl 

1 5p f* 

C2 

0. 1 

CO 

0. 1 

C4 

1 5p f 

C5 

0. 1 

C6 

0. 1 

C7 

0. 1 

C8 

15pf 

C9 

0. 1 

CIO 

0. 1 

• 1 5p f 

Cll 

C 12 

0. 1 

C 13 

0. 1 

C 14 

1 5p f 

CIS 

0. 1 

C16 

0. 1 

C 17 

1 5p f 

C 18 

0. 1 

C 19 

0. 1 

C20 

15pf 

C21 

0. 1 

C22 

0. 1 

C23 

15pf 

C24 

0. 1 

C25 

0. 1 

C26 

15p f 

C27 

0. 1 

DIODES 



C28 

0. 1 

C29 

0. 1 

C30 

0. 1 

C31 

0. 1 

C32 

0. 1 

C33 

0. 1 

C34 

0. 1 

C35 

0. 1 

C36 

0. 1 

C37 

0. 1 

C38 

0. 1 

C39 

lOOpf 

C40 

0. 1 

C41 

0. 1 

C42 

0. 1 

C43 

lOOpf 

C44 

0. 01 

C45 

0. 01 

C46 

0. 01 

C47 

0. 01 

C48 

0. 1 

C49 

0. 1 

C50 

0, 1 

C51 

0. 1 

C52 

0. 1 

C53 

0. 1 


The selected diodes must supply 0.7 forward biased voltage. 


location 

mode 1 

D1 

GE 914 

D2 

GE 914 

D3 

GE 914 

D4 

GE 914 

INTEGRATED 

CIRCUITS 

location 

model 

IC1 

AD509JH 

ICS 

AD509JH 

IC3 

AD509JH 

IC4 

AD509JH 

IC5 

AD509JH 

IC6 

AD509JH 

IC7 

AD509JH 

ICS 

AD509JH 

IC9 

AD509JH 

IC10 

RC4200ND 

IC1 1 

DG509CJ 

IC 12 

R5402 E 146 

IC13 

MC1408L8 

IC 14 

MC1408L8 

IC 1 5 

NB2S09N 


iUlb 

IC17 
IC 18 
IC19 
IC20 


r,.N7432J 

SN7474N 

D50026CN 

Db002toCN 


JUMPERS 

jl takes the multiplexer output to the PNC connector 

j 2 takes the output of the squaring circuit to the BNC connecto- 

J3 joins both paths of ground 
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Abstract 

The potential uses of charge transfer devices (CTD's) in 
pattern classification operations are explored. The needs for 
a hardware-based pattern classifier are established, and a ma- 
trix multiplication subsystem based upon a sum of products CTD 
is presented. An evaluation process for sum-of-products de- 
vices (particularly analog-analog correlators) is developed, 
and the feasibility of employing. a particular device in a pat- 
tern classifier is determined. Finely, the possible impact of 
future trends in technology is considered. 

I. Introduction 

Recent technological innovations are making general pur- 
pose computers cheaper and more accessible. Witness, for ex- 
ample, the dramatic increase in computational complexity avail- 
able per dollar in just the last five years. These same tech- 
nological innovations are making instrumentation packages sim- 
pler to use, more computationally dense, and much less expen- 
sive. It thus is becoming more and more reasonable to talk 
about special purpose, dedicated pattern recognition 
equipment. 

We can discuss only a small subset of the "pattern recog- 
nition problem" in this context, since that larger problem is 
far from well defined, much less solved, and "special purpose 
equipment" implies that we are trading away flexibility in 
exchange for speed and/or simplicity of use. We have chosen to 
deal with the problem of multi spectral satellite image classi- 
fication. Under certain assumptions, this problem can be con- 
sidered well defined, and a pressing need exists for special 
equipment which can deal rapidly with the vast amounts of data 
coming from satellites every minute. 

NASA has been and continues to be concerned about the fact 
that present (general purpose computer-based) techniques are 
too slow and too expensive to begin to deal with more than a 
tiny fraction of the LANDSAT data which are currently 
available. This paper is one of the results of an ongoing 


study conducted by NASA to investigate technologies which might 
contribute to the solution of this data processing bottleneck. 
This paper discusses the use of metal-oxide>semiconductor (MOS) 
technology in the construction of special purpose equipment for 
pattern classification. 

V 11. Device Background 

Charge transfer devices may be divided into two classes: 
the "bucket brigade device" (BBD) and the "charge coupled de> 
vice" (CCD) with various subclasses within the major groupings. 
The two types of CTD's'differ in the manner in which charge is 
stored and transferred from cell to cell and have slightly 
differing performance characteristics. Most CCD’s have lower 
noise figures and higher transfer efficiencies (the percentage 
of charge in the original cell which is transferred to the new 
cell) than BBD's. 

When used as analog signal processors, charge transfer de- 
vices may be employed to obtain complex functions of the input 

waveform. One such function is Fa-h.,, where the b's are 

i 1 

samples of the input waveform and the a's are weighting 
coefficients. This function is found in recursive filtering, 
correlation, convolution, and a number of other operations. 

As shown in Fig. 1 and photographed in Fig. 2, the cor- 
responding cells of two CTD delay lines can be connected to 
multipliers and the multiplier outputs then summed. In this 

case the result is again £a.b,., but the weighting coefficients 

i 1 1 

are determined by the data stored in the second CTD and may be 
changed simply by clocking in new data. 

Such variable t?o weight devices may be used as sampled 
correlators of continuous analog signals. More important to 
this project, however, is the fact that, since they operate on 
discrete data samples, variable tap weight devices may be used 
to generate the vector dot product, 

n 


Where n is the dimension of the vector and may be as large as 
the number of cells in the CTD. This product is obtained 
Simply by loading one vector into one side of the CTD and the 
second vector into the other. The answer thus obtained is the 
result of one row by column operation of a matrix 
multiplication. 


- Unlike conventional filtering application, in which a use- 
ful new result is available each clock cycle, dot product oper- 
ations with two arbitrary vectors require that the entire CTD 
be loaded before a useful answer Is produced. Thus, for a 
typical 16 component vector, 16 clock cycles are required to 
loadythe device. However, 5-Mhz clock rates are quite reason- 
able, making It possible to perform 16 multiply and add oper- 
ations in 3.2 usee. 

III. Pattern Classification Hardware 

** i 

In the case of multispectral image classification, the in- 
put X is a vector resulting from measurements of light inten- ■ 
sity in several different spectral ranges. Each vector X 
corresponds to a single point (pixel) in a scene. A typical 
LANDSAT scene consists of an array of over one million ordered 
pixels. 

It has been shown^ that, for a given class, all pixels 
belonging to that class may be reasonably described by a multi- 
variate normal distribution. With this assumption, the prob- 
ability that a veetbr X belongs to a class i is 

P ^ X) n (2n N/2 )(|M 1/2 ) expC '^ X_w i )T 

where ^ and u^ are the covariance matrix and mean vectors, 
respectively, which describe the statistics of class i. 

Taking the logarithm of the probability gives a discrimi- 
nant function 


9, (X) A 4DtP(1|X)] « -1/2(X- Ui ) T X^X-m,) 

+ * n IZ-f | _1/Z + w(2it‘ N/2 ) 


Since the logarithm function is monotonic, the class 
having the largest discriminant function for a given 
measurement X will also be the class having the largest 

probability P(i|X) that X belongs to that class. tn(2Ti“ N ^ 2 ) 

is a constant for all classes and therefore does not contribute 
to discriminating one class from another. Furthermore, the 

term *n|£|”^ 2 needs to be computed only once for each class. 

(X-u^) T I^(X-w.j). however, must be computed for each of the 


millions of measurements made in an image. Consequently, this 
matrix computation with a general purpose computer is very 
time consuming. 

Figure 3 shows a block diagram for a hardware configura- 
tion of a system for classifying multispectral data. Data may 
come direct ly from a sensor array in analog form, or for pur- 
poses of testing, from a digital data source. The microcom- 
puter is the control element for the system. In a training 
mode, it derives the statistics which describe the various 
classes. In classification mode, the microcomputer loads 
those statistics into an array of parallel CTD classifiers, 
controls steering of data input to those classifiers, and 
examines their outputs. 

The individual classifiers are shown in Figs. 4 and 5. 

In Fig. 4, each of the row-column dot product operations is 
performed in a charge transfer device. The outputs are 
multiplexed together and fed to one more TTD for the post- 
multiplication dot product. 

In contrast. Fig. 5 depicts a hardware simplification 
which also results in increased speed, since it eliminates the 
multiplexer and a delay. This simplification is made feasible 
by the diagonal symmetry of the covariance matrix, which may 
be transformed into upper triangular form. 

The matrix operation shown in Fig. 5 can be decomposed 
into cellular substructures as shown in Figs. 6 and 7. At the 
conclusion of the training mode, the processor loads one row 
of the covariance matrix into the first-in-first-out memory 
associated with each cell. The hardware then takes over and 
under control of the clock generator, performs the entire 
discriminant function computation. 

On a pixel-by-pixel basis, the output of the individual 
hardware classifiers is digitized and read by the processor, 
which then classifies the pixel as belonging to the class 
whose discriminant function was maximized. Using 128 CTD's, 
a pixel described by 16 multispectral measurements may be 
classified into one of nine categories in 3.2 usee. 

IV. In Situ Cell Qualification 

The critical element in the pattern classifier system is 
the charge transfer device that performs the row-column 
multiply. Extensive testing has been performed on prototype 
units which have recently become available from semiconductor 
manufacturers. 

The hardware test station shown in Fig, 8 has been im- 
plemented. It tests individual devices in a cellular 
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structure similar to the structures used in the classifier. 
Typical results of one of the tests on one of the devices are 
shown in Fig. 9. The figure shows the aggregate linearity 
and offset for a Reticon 5402, a 16-element analog-analog 
sum of products device. Plotted is the computed output in 
expanded scale and inverted decimal form) versus the B input 
values', where all of the B cells are filled with the 
fractional numbers indicated. Similarly, the A cells are 
filled with the numbers indicated, and these point values are 
connected to form families. It is apparent from this figure 
that there is a small offset, the point where the curves all 
intersect. There is also a small nonlinearity, where the 
points fall off the curve. The rotated appearance of the 
curve is caused by the small offset in the B side being 
multiplied and suitmed by the data in the A side. These test 
results demonstrate adequate linearity and offset at the 8 
equivalent bit input and 8 equivalent bit output to 
continue further development. Figure 10 is a photograph of 
the test system showing the chip under test, the micro- 
computer and the associated da.ta conversion circuitry. 

* V. Conclusion 

This paper has shown one method of implementing dedicated 
hardware for pattern classification. Recent technological 
developments have made such classifiers feasible using 
sampled analog processing, Test results have indicated that 
prototype development should continue. It is expected that 
continued technological improvement will lead to more 
compact, lower power, and even faster system configurations. 
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Abstract 

The potential uses of Charge Transfer Devices 
(CTDs) in pattern classification operations are 
explored* The needs for a hard ware -based pattern 
classifier are established, and a matrix multi- 
plication subsystem based upon a sum-of -products 
CTD is presented. Applications of the subsystem 
to the classification of multi-modal Gaussian 
distributions in general and to LANDSAT data 
processing in particular are discussed. Finally, 
the potential impact of this technology on 
satellite data processing methodologies is 
discussed. 

Key words: Gaussian Classifier, Charge coupled 

device 
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1. Introduction 

Recent technological innovations are making 
general purpose computers cheaper and more 
accessible. Witness, for example, the dramatic 
technological increase that has been made in just 
the last five years. These same technological 
innovations are making instrumentation packages 
simpler to use, more computationally dense, and 
much less expensive. It is thus becoming more 
and more reasonable to talk about special purpose, 
dedicated, pattern recognition equipment. 

We can discuss only a small subset of the 
"pattern recognition problem" in this context 
since that larger problem is far from well defined, 
much less solved, and "special purpose equipment" 
implies that we are trading away flexibility in 
exchange for speed and/or simplicity of use. We 
have chosen to deal with the problem of multi- 
spectral satellite image classification. Under 
limiting assumptions, this problem can be 
considered well defined and a pressing need 
exists for special equipment which can deal 
rapidly with the vast amounts of data coming 
from satellites every minute. 

NASA has been and continues to be concerned 
about the fact that present (general purpose 
computer-based) techniques are too slow and too 
expensive to reduce to classified images more 
than a tiny fraction of the LANDSAT data which is 
currently available. This paper is one of the 
results of an on-going study conducted by NASA to 
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investigate technologies which might contribute to 
the solution of this data processing bottleneck. 

Of particular interest to NASA are 
technologies which may lead to flyable "on-board" 
processors, units on the satellite which can 
classify a multi-spectral image in real-time and 
transmit to the ground only the 'classified image. 

A unit must meet several requirements before it 
can be placed in such an application. First, it 
must be. capable of dealing with the analog data 
directly as it comes from the sensors. Second, 
since the unit is to be placed on a satellite, it 
must consume little power, be light in weight, and 
be highly reliable. Finally, the unit must be 
programmable from the ground and capable of 
deriving its own classification parameters. 

This paper discusses the use of Metal-Oxide- 
Semiconductor (MOS) technology in the construction 
of special purpose equipment for pattern classifi- 
cation. The computational function of individual 
sum-of -products chips is first described, then, 
a scheme for the organization of the chips into 
a pattern classifier is shown. 

The potential of on-board classification has 
both possible gains and hazards associated with 
it. With the obvious benefits of timely data 
availability comes the potential of the unavaila- 
bility of the raw data for further processing. 

This difficulty is discussed in Section 


2 . Device Background 

Charge Transfer Devices may be defined for. 
the purposes of this paper as devices which move 
charge linearly in synchronism with a clock. If 
the charge is quantized in a binary manner, CTDs 
may be used as digital delay lines or as shift 
register memories. It is, however, the ability of 
CTDs to move analog data that has resulted in 
their widest application. They have been used to 
acquire analog video data and to process analog 
data from other sources. Special classifier hard- 
ware fits into this last category. 

Charge transfer devices nay be divided inic 
two classes: the "bucket brigade device" (EBD) 

and the 'fcharge coupled device" (CCD) with various 
subclasses within the major groupings. The two 
types of CTDs differ in the manner in which charge 
is stored and transferred from cell to cell and 
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have slightly differing performance characteris- 
tics. 

Host CCDs have lower noise figures and higher 
transfer efficiencies (the percentage of charge in 
the original cell which is transferred to the new 
cell) than BBDs. 

When used as analog signal processors, charge 
transfer devices may be employed to obtain complex 
functions of the input waveform. One such 

function is Ea^b^ where the v b*s are samples of the 

Input waveform and the 'a's are weighting 
coefficients. This function is found in recursive 
filtering, correlation, convolution, and a number 
of other operations. 

Buss 1 shows a simple means for implementing 
"fixed tap weight" devices where the tap weights 
are the weighting coefficients of the Ea.bj 

function and are fixed at the time of manufacture. 
The tap weights are realized by splitting the 
transfer electrodes of one clock phase of the CTD 
in the ratio (1 * ) : (1 - a^ ) where a^ is the " 

ith desired coefficient. 

In an alternate and potentially somewhat more 
useful approach, as shown in Figure 1 and photo- * 
graphed in Figure 2, the corresponding cells of 
two CTD delay lines can be connected t£> multipliers 
and the multiplier outputs then summed. In this 
case the result is again Ea^, but the weighting 

coefficients are determined by the data stored in 
the second CTD and may be changed simply by 
clocking in new data. 

Such variable tap weight devices may be used 
as sampled correlators of continuous analog 
signals. More important to this application, 
however, is the fact that since they operate on 
discrete data samples, variable tap weight devices 
may be used to generate the vector dot product, 
n 

X*Y s Z X.Y. where n is the dimension of the 
i=l x 1 

vector and may be as large as the number of cells 
in the CTD. This product is obtained simply by 
loading one vector into one side of the CTD and 
the second vector into the other. The answer thus 
obtained is the result of one row by column opera- 
tion of a matrix multiplication. 

Unlike conventional filtering applications, 
in which a useful new result is available each 
clock cycle, dot product operations with two 
arbitrary vectors require that the entire CTD be 
loaded before a useful answer is produced. Thus 
:or a typical 16 component vector, 16 clock cycles 
are required to load the device. However, 5 Mhz 
clock rates are quite reasonable, making it 
possible to perform 16 multiply and add operations 
in 3.2 us. 


3. Pattern Classification Hardware 
2 

It has been shown that for the purpose of 
LANDSAT data classification, all pixels belonging 
to a given class may be described (typically), by 
a multimodel multivariate distribution. The 
multimodal distribution may be adequately 
decomposed into an aggregate of Normal distribu- 
tions. Classification then consists of determining 
which of several Normal distributions a particular 
pixel is most likely to belong to and assign the 
pixel to the class having that distribution. With 
this assumption, the probability that a vector X 
belongs to a class i is 

P(w i |X) * 

1 

(2w) W/2 (|C i | 1/2 ) EXP(-l/2(X-M i ) T C 1 "^(X-M i )) 

Where C^ and u^ are the covariance matrix and mean 

vectors respectively which describe the statistics 
of class 1. 

The usual definition of the Normal distribu- 
tion includes a term representing the a-priori 
probability P(w^) that a sample X belongs to a 

particular class w^. Experience has shown that 

very satisfactory results can be had by treating 
all a-priori probabilities as equal. If this is 
the case, then for the purposes of classification, 
the a-priori probabilities may be neglected. 

Taking the logarithm of the probability gives 
a discriminant function 

g i (X) = inP(w. | X) = 

-1/2(X-u 1 ) T C 1 ' 1 (X-m.) ♦ tn|C.|’ l/2 + »n(2n‘ N/2 ) 

Since the logarithm function is monotonic, 
the class having the largest discriminant function 
for a given measurement X will also be the class 
having the largest probability P(w.JX) that X 

belongs to that class. 

£n(2ff~ N ^ 2 ) is a constant for all classes and 
therefore does not contribute to discriminating 
one class from another. Furthermore the term 

Inlc)’ 1 ^ 2 needs to be computed only once for each 
class. 

(X-u i )T C^CX-u.) however must be computed 

for each of the millions of measurements mad® in 
an image. Consequently, this matrix computation 
with a general purpose computer is very time 
consuming. 

Figure 3 shows a block diagram for a hardware 
configuration of a system for classifying multi- 
spectral data. 

Data may 'come directly from a sensor array in 
analog form or, for purposes of testing, from a 
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■»*>d« , it derive; t :.t*t . • i which descril* 
t hr various clastec, in clacr if icwt ion modt , the 
microcomputer lead* those LtutinticL intv «r> 
array of parallel C7b classifiers, controls 
steering of the data input to those classifiers, 
and examines their outputs. 

The Individual classifiers are shown in 
figures *. and S. In figure 4 V each of the row- 
column dot product operations is performed in a 
charge tranter device. The outputs are multi- 
plexed together and fed to one more CTD for the 
postmultiplication dot product. 

In contrast v figure b depicts a hardware 
simplif icat ion which also results in increased 
speed since it eliminates the multiplexer and a 
delay. This simplification is made possible by 
the following argument: 

By the constructed diagonal symmetry of the 

C matrix, the product z * [X^J |C* A ) (X) may be 
rewritten as 

z * (X* J (A) * (A J [ X] where A is upper triangular 
then . T 
z » |XV)IAXJ 

• |y T im 

. r’. 

The matrix operation shown in figure b can be 
decomposed into cellular substructures as shown in 
figure 6. If 8 features are assumed, then 8 of 
the sum-of -product s structures in figure t are 
needed -• one for each row column operation. 

At the conclusion of the training mode, the 
processor loads of the covariance matrix 
associated with each class into the covariance 
memory associated with each cell. If 8 possible 
Classes are assumed, this memory is 64 x 8 bits 
as shown. During this time, the X-u vector for 
each class is loaded into the X-w RAM also. 

At this point, the hardware controller takes 
over and performs the discriminant function 
computation. The outputs of all sum-of -product s 
cells are summed and passed through the analog 
to digital converter whose output is read by the 
processor as shown in figure 7. This operation 
is repeated for each class simply by stepping to 
the next group of covariance matrix rows and X-u 
vectors in RAM. The processor then classifies the 
pixel as belonging to the class where discriminant 
function was maximized. 

Operating the classifier in this fashion with 
longer memories holding ail of the statistical 
information allows the use of only a single 
classifier without the need to reload the 
covariance infc/imat ion for each discriminant 
function calculation. In this manner a significant 
reduction in hardware over the use of a separate 
classifier for each class is realized with only a 
slight reduction in operating speed. 


I jri Da*.- If ic >! r i 

** have ghuwv. in this paper an architecture 
which swikes feasible the possibility of on-board 
clatsif icat ion. An on t*oard classifier offers 
significant potential gains in performance of the 
satellite system; data would be available to the 
user in minutes rather than months. 

There are significant logistical and technical 
problems which must be overcome before these 
benefits could become reality. In this section, 
we demonstrate only a few and their potential 
solutions. 

A Scenario 

We will make this demonstration through a 
scenario of how s typical classification might be 
performed : 

(1) A county agricultural agent reserves the 
satellite for its next pass over. In so doing, 
he specifies the coordinates of some areas known 
to be corn, soybeans, and cotton. 

(2) The coordinates of these training sets are 
transmitted to the satellite. As the satellite 
passes over, image data is acquired and stored. 

The on-board classifier performs a cluster 
analysis on the training sets and derives a 
Gaussian fit for each cluster. 

In an alternative proposed system 3 the satel- 
lite clusters the entire scene, transmits the 
cluster statistics, and for each pixel, transmits 
the number of the cluster to which that pixel is 
assigned. 

(3) Once appropriate statistics have been derived 
to describe training sets, the classifier described 
in section 3 is initialized by loading the 
statistics into the RAMS, and the data is then 
classified as belonging to one of the classes 
identified as corn, soybeans, or cotton. The 
results of the classification are then encoded 

and transmitted to the ground. 

(u) The agriculture agent then can receive a false 
colored map of the area or a digital tape with the 
classification results. 

It should be noted that this scenerio has 
passed over a significant amount of pre-processing 
which must be done to the sensor output prior to 
classification, including correcting for geometric 
distortion. 

On board classification provides a tremendous 
potential benefit since it makes reasonable direct 
user interaction with the satellite; and provides 
data for the user in expeditious time. The one 
factor which some users may consider detremer.tai 
in such a system is the fact that no longer doe* 
the ground user have the raw data to mull over at 
his leisure. 

This factor does open up a new area of study, 
for in those instances when the user has both a 
computer and the time to study the image, he ray 
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at the on-bvard classifier, !t Is possible 
. . thli circumstance to i-prove classifier 
-rt rune# by using these data. •- jre currently 
tuning this proble-n and wil 1 be publishing our 
.• .Its in the near future. 


5. tonclus ion 

This paper has shown one method of 
implementing dedicated hardware for pattern 
:lassif icat ion. Recent technological develop- 
ments have trude such classifiers feasible using 
14 -iplod analog processing. 


fi 


Test results have indicated that prototype 
leveiopment should continue. It is expected that 
rontlnued technological improvement will lead to 
rri* •jn; ict , lower power, and even faster systr- 
:onf igurat ions . 
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Abstract 

The potential unci of Charge Transfer Devices 
(CTDg) in pattern claaaif icat ion operation* are 
explored. The needs for a hardware-based pattern 
classifier are established, and a matrix multipli- 
cation subsystem based upon a sun of products CTD 
is presented. An evaluation process fot sun of 
products devices (particularly analog-analog cor- 
relators) is developed, and the feasibility of 
employing a particular device in a pattern classi- 
fier is determined. Finally, the possible impact 
of future trends in technology is considered. 

1 . Int roduct ion 

Recent technological innovations are making 
general purpose computers cheaper and more accessi- 
ble. Witness, for example, the dramatic Increase 
in computational complexity available per dollar 
in Just the last five years. These same techno- 
logical innovations are making instrumentation 
packages simpler to use, mor^ computationally 
dense, and much less expensive. It is thus becom- 
ing more and more reasonable to talk about special 
purpose, dedicated pattern recognition equipment. 

We can discuss only a small subset of the 
"pattern recognition problem" in this context since 
that larger problem is far from well defined, much 
less solved, and "special purpose equipment" 
implies that we are trading away flexibility in 
exchange for speed and/or simplicity of use. We 
have chosen to deal with the problem of mult taper- 
tral satellite image classification. Under certain 
assumptions, this problem can be considered will 
defined and a pressing need exists for special 
equipment which can deal rapidly with the vast 
amounts of data coming from satellites eve r\ 
minute . 

NASA has been and continues to be concerned 
about the fact that present (general purpose 
computer-based) techniques are too slow and ten 
expensive to begin to deal with more than a tiny 
fraction of the LANDSAT data which is currently 
available. This paper is one of the results of an 
ongoing study conducted by NASA to invest ipati 
technologies which might contribute to the solution 
of this data processing bottleneck. 

This paper discusses the use of met al-oxldt- 
semiconductor (MOS) technology in the construction 
of special purpose equipment for pattern 
classl f icat Ion. 
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2 . Device Background 

Charge transfer devices may be divided into 
two classes: the "bucket brigade device" (BbD) 

srd the "charge coupled device" (CCD) with various 
subclasses within the major groupings. The two 
types of CTDs differ in the manner in which charge 
is stored and transferred from cell to cell and 
have slightly differing performance characteristics. 

Host CCDs have lower noise figures and higher 
transfer efficiencies (the percentage of charge in 
the original cell which is transferred to the new 
cell) than BHDs. 

When used as analog signal processors, charge 
transfer devices may be employed to obtain complex 
functions of the Input waveform. One such 
function lajbj where the 'b's are samples of the 
Input waveform and the *a's are weighting coeffi- 
cients. This function is found In recursive 
filtering, correlation, convolution, and a number 
of other operations. 

As shown in Figure 1 and photographed In 
Figure 2, the corresponding cells of two CTD delay 
lines can be connected to multipliers and the 
multiplier outputs then summed. In this esse the 
result is again Lsjbj, but the weighting coeffi- 
cients are determined by the data stored in the 
second CTD and may be changed simply by clocking 
In new data. 

Such variable tap weight devices may be used 
as sampled correlators of continuous analog sig- 
nals. More Important to this project, however, is 
the fact that since thev operate on discrete data 
samples, variable tap weight devices may be used 
to generate the vector dot product, 
r 

X • ^ • l XjVj where r» is the dimension of tin- 

vector and tnav he ar large as the numbet of cells 
In the CTD. This product is obtained slmplv bv 
loading one vector Into one side of the CTD an/! 
the second vector Into the other. The answer thus 
obtained is the result of one row by column 
operation of a matrix rult lpl icat Ion. 

Unlike conventional filtering appl icat ic r. , it. 
which a useful new result Is available eac v ch * » 
cycle, dot product operations with two arbltrarx 
vectors require that the entire CTD be loadt d 
before a useful answt r Is produced. Thus for a 
typical lb component vector, lb clock cvcles art 
required to load the device. However, 5 Khz clt» 
rates art quite reasonable, making It pcsslll* t 
perlcr- It multiple and add operations 1: j.. . . 
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1 . 

In the (dim of mu) 1 1 »«| « < t v al !> •» * 1 «»• 1 f 1 - 

mtlon, I hr inpul , X, In a v» » 1* i resulting Inn 
nnurrmcnt n of light intensity In m-vhii) cl I f f • r - 
ml spectral ranges, Each vector X correspond* 
to s single point (pixel) in s scene. A typical 
LANUSAT scene consists of an array of over one 
Billion ordered pixels. 

• 

It has been shown (l) that for a given class, 
all pixels belonging to that class stay be reason- 
ably described by a multivariate normal distribu- 
tion. 01th this assuaptlon, the probability that 
a vector X belongs to a class 1 is 

i 

P(i/X> - i 7JT T77 EXM-l/JU-u.) 1 C l (X-w.)) 

Clw'>(|l/"> ‘ 

Where Ij and bj are the covariance matrix 
and mean vectors respectively which describe the 
statist les of class 1 . 

Taking the logarithm of the probability gives 
a discriminant function 

« t <X) • ln(P(i/X>) • -I/2 <X-u 1 ) T ijNx-Uj) 

4 Cn | r 4 | 1 / 2 4 tn(2r~ h/! ) 

Since the logarithm function is monotonic, the 
class having the largest discriminant function for 
a given measurement X will also be the class 
having the largest probability P(l/X) that X 
belongs to that class. 

-N / 2 

ln(2ir ) Is a constant for all classes and 
therefore does not contribute to discriminating 
one class from another. Furthermore, the term 

£n|l p/2 needs to be computed only oi.ee for each 

class. 

Ij^(X-bj) however must be computed 

for each of the millions ol measurements made in 
an image. Consequently, this matrix computation 
with a general purpose computer is very tln.< 
consun) ng. 

Figure 3 shows a block diagram for a hardwan 
configuration of a system for classifying multi- 
spectral data. 

bats nwiy come dlrectlv from a aensor array in 
analog form ot , for purposes of testing, from a 
digital data source. The microcomputer is the con- 
trol element for the avstem. In a training modi , 
it derives the statistics which describe th< 
various classes. In classification mode, the 
microcomputer loads those statistics into an nrn»N 
of parallel CTD classifiers, controls steering of 
data Input to those classifiers, and examines their 
output!. . 


If I ' . ' l.i • If lev* Mi r K ..t It | <* * 

uf • •* It I . , I • 4, I III it I I I hi I !*• — 

it. In? d* t pr« dm t operation* 1 •• perf«»rr* d In <* 
cl.ir# it*. f«i d'Viit, lU cut puts um r»u)t 1- 
( )• r. • t # • thrf atid f*d t< on« raft CTl) fe r tin 
postmull lf llc.it Ion d- t produwt. 

In contrast, flgun 3 depicts a hardware 
airpl 1 f lest ion which also results in increased 
speed since it eliminates the multiplexer and a 
delay. This simplification Is made feasible by 
the d'ago*-*1 symmetry cl the "v/arlance matrix 
which may be transferred ln»o upper triangular 
form. 

The matrix operation shown in figure 3 can 
be decomposed into cellular, substructures as 
shown In figures 6 and 7. At the conclusion of 
the training mode, the processor loads one row of 
the covarlanca matrix into the f lrst-ln-f irst -out 
memory associated with each cell. The hsrdwart 
then takes over and under control of the clock 
generator, performs the entire discriminant func- 
tion computation. 

On a pixel by pixel basis, the output of the 
individual hardware classifiers are digitised and 
read by the processor, which then classifies the 
pixel as belonging to the class whose discriminant 
function was maximized. Using 128 CTDs, a pixel 
described by If mult 1 space ral measurements may be 
classified into one of nine categories in 3.2 lit. 

4. In Situ Cell Qualification 

The critical element in the pattern classlfiet 
system is the charge transfer device which periormt 
the row - column multiply. Extensive testing hat. 
been performed on prototype units which have 
recently become available from semiconductor 
manufacturers . 

The hardware test station shown in flgun 8 
has been implemented. It tests individual drvins 
in a cellular structure similar to the structures 
used in the classifier. 

Typical results of some of these tests an 
shown in figure 9. Shown *re results of testing 
three different devices, the Helicon AAC-32, a 
32 element unit, the 3403, a modified AAC-32, and 
the 3402, a 16 element unit. The figure shows 
output numerical computations in families for 
varying input number?-. 

in going from the AAC-32 to tht 3403, Helicon 
significantly improved the numerical rangi of 
calculat ions . 

3. Conclusirr 

This paper has shown one method of implerir t- 
lng dedicated hardware for pattern c 1 ass 1 1 i c .it i 
Recent technological developments have rvid* nui *» 
classifiers feasible using sar.pli d analog 
processing . 
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ABSTRACT 


This paper addresses a particular application, that 
of tracking moving objects with a television camera, (1, 2, 

5, ....) and deals in this context with the interactions 
between motion, image gradients, and image representations. 

A representation for moving images in terms of Taylor's 
series expansions is developed, and application of this ex- 
pression is shown to suffer from the assumption that images 
are continuously differentiable. A simple numerical example 
is given, and the velocity • segmentation work of Pennema 
and Thompson is explained in this context. 

It is concluded that motion information results only 
from the motion of edges and that not understanding this 
principal can lead to problems. 

Correlation trackers, and in particular, a tracker 
developed by Pitts', are shown to, in fact, correlate the 
* motion only of edges. 

Finally, in the appendix, it is shown how Fitts' tracker 
can be derived from first order Taylor's series assumptions. 

Key words: Tracking, gradients, motion. 


I. INTRODUCTION 

Traditionally, the intensity function representing the bright- 
ness at a point in an image is considered to be a two dimensional 

♦ 

continuous function, I(x,y) . Image operators derived using this 
model are then discretized by setting Ax and Ay to one in the finite 
increment model. 

We will extend this model to consider the possibility of motion 
within a scene by allowing the intensity function to vary in x,y, and 
t. The images will be modeled in the context of digitized television 
images, therefore the t will be discretized to the frame -to -frame 
time interval. 

We will examine a particular model for the image, the Taylor 
series expansion of the intensity function, and it will be shown how 
this expansion leads to a mechanism for segmentation and velocity 
detection. The capability of segmentation results from the concept 
that all the pixels having the same velocity will belong to the same 
region. Some work in this area has been done by Fennema and 

Thompson [3], and is extended here. A second method of determining 
pixel velocity will then be considered, a correlation tracker. This 
method was introduced by Fitts [4] . The correlation tracker assumes 
that at least some segmentation has already been done and, in fact, 
this method estimates the velocity of a group of pixels. We will 
then compare the velocity estimation accuracies of these two methods 

and come to some rather predictable conclusions that better tracking 
can be done if some segmentation has already been done. (See {6, 9, 

II, 13) for related work) . 


The fundamental weakness of both of these approaches will then 
be noted. We will generalize this to a philosophy for dealing with 
digitized images . , 

2. THE TAYLOR SERIES MODEL FOR TIME-VARYING IMAGES 


For purposes of clarity, we will at first describe the intensity 
function 1 as a function of a single variable, x, and consider the case 
of shifting I(x) . 

I(x) thus, represents in one-dimension the grey scale values of 
the object to be tracked. It will be the same function in two sequen- 
tial frames, but will be displaced in x. 

2 

Consider the following example, I(x) * x . This is shown in 
Figure 1 , 



At some point on the x axis, x^ , we measure I o (x^) from the 
first frame, and later, at this same point x-^ we measure 1^ (x^) from 
the second frame. But the two functions are the same except for a 
shift in the axis , so 


I 0 (x-6) - I x (x) . 


original page ib 

OP POOR QUA/jj® 


( 1 ) 


( 2 ) 


Expanding I 0 (x - 6) about x^, we get* 

* 0 l*l-*> “ " 6I o< x l> + 7* *o “ 

and from the equality (1) we find that 

hf*i> “ *«<*!-« ' 1 o< x i > - + Ov y 


and finally 


81. 


i x (*x> “ i 0 <x i ) “ < x l> 6 + 


dx 


3 2 I Q ,2 

j ( x l^ 

a x z 1 2 


(3) 


I 0 < x i) is the value of the intensity function measured at point 
x^ in the first frame, and likewise, 1^ (x^) is measured in the sec- 
ond frame. Thus, if we, can compute or accurately estimate the gra- 
dients in the first frame, we can solve the above equation for the 
displacement 6 . 

Now, let us consider the extension of this concept to two di- 
mensions. Assume we have a function I o (x,y) and a second function 
I^(x,y) which results from shifting I Q (x,y) through $ x and 6 y . That 
is , 

h < x »y> “ * 0 < x - 6 x »y~V • 

Expanding I 0 (x-6 xt y-6 ) at a point (x^y^, we get 


VW^i-V 


-f 


8I £ .« y V 


dx 


+ 


h 


C3 2l 0 (xi,yi) 




+ 


2 3 2l b (x 1 ,yi> 

3x3y 


.6 6 

x y 


+ 


3 I o^ x l ,y l^ 


3/ 



(5) 


However, we know from (A) that this is equal to I^x^y^ which 

is measured in the second frame. 


Now, let us consider the following notatlonal simplifications 
and define; 


_ a V*i-yi>. 




ax 


v 

G ■ 
xx 


3 V x l’ y l> 


ax' 


ay 


G yy ~“17 G xy 


8 1 0 < x 1 >yi) . 

axdy 


and AI - I^(x^,yj) - I 0 ( x i.yi>» and this reduces to 

AI - 6? + 6^ +.G 6 6 - G 6 - G 6 (6) 

A 2 x 2 y xy * y x x y y v ' 

Thus, a single pixel (x,y) substituted into (6) will yield. a 

quadratic equation in two unknowns , 6 and 6 . However, all pixels 

x y 

having the same velocity will have characteristic equations whose 
solutions pass through the same point . 

In practice, due to noise, one finds that the solutions pass 
through nearly the same point, so that a clustering technique may 
’ be used to find the best approximation to that point of common inter- 
section, thus yielding an algorithm for finding frame-to-frame displacemej 
2.2 Several Numerical Examples 

Let us consider the ability of this representation to determine 
the displacement of a simple signal. Assume displacement only occurs 
in the x direction so we may use the simpler ' one dimensional form 


AI - k 


G *6 2 

XX X 


- G 6 

X X 


6 

x 



+ 2 Gxx AI 


or 


( 


i [ 

mm 


or , In the case G ■ 0 , we use 

^ A 


* . iAI 

x G 
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First, we apply the ‘technique to I » x between -6<x<6 






After experimenting with this algorithm with several different 
signals, we observed the following: ' 

A 

1) For signals continuous over the domain, 6 is computed 
exactly. For example, 

S - x 2 

V 

y - <*-« x > 2 

o 

Computing this function analytically for s - x yields an exact 
result. That is 6 * 6 X for any 6*. We need to point out that 
S(x) is continuous in all derivatives and defined over the tire 
interval from -a to a for any a. 

2) A test of the predictor was made numerically , again using 
2 

S « x , over the interval -6 to-* 6. With Lx • 1, the following 
numerical values . * 


x 

-6 

-5 

-4 

-3 

~2 

-1 

0 

1 

2 

3 

4 

5 

6 

S(x) 

36 

25 

16 

9 

4 

1 

0 

1 

4 

9 

16 

25 

36 

'y(x) 

64 

49 

36 

25 

16 

9 

4 

1 

0 

1 

4 

9 

16 

as 

n§x 

-12 

-10 

-8 

-6 

-4 

-2 

0 

2 

4 

6 

8 

10 

12 

w 

12 

10 

8 

6 

4 

2 

0 

-2 

-4 

-6 

-8 

-10 

-12 

! y-« 

28 

24 

20 

16 

12 

8 

<4 

0 

-4 

-8 

-12 

-16 

-20 

..W(y-s) 

336 

240 

160 

96 

48 

16 

0 

0 

16 

48 

96 

160 

240 

-IW(y-s)« 

1456 













m ■ 

144 

728 

100 

64 

36 

16 

4 

0 

4 

16 

36 

64 

100 

144 


= 1456 
r 3s 2 ”72* 

1 l3xl 


This rather remarkable result, an exact answer, can be attributed 
to the fact that x is a nicely behaved function. Let us try it 
with some other functions which are not quite so nice. 


X 

-6 

-5 

-4 

-3 

-2 

lL 

0 

1 

2 

3 

4 

5 

6 

a V 
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0 

0 

2 1 

4 

4 

4 

2 

0 

0 
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0 

0 

0 
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0 

0 

0 

2 

4 

4 

4 

2 

a 

0 

0 

3s 

Sx 

0 

0 

, 1.18 

2 

1.18 

0 

- 1.18 

-2 

- 1.18 

0 

0 

r 

0 

0 

w 

0 

4 

0 

- 1.18 

-2 

- 1.18 

0 

- 1.18 

2 

1.18 

0 

0 

0 

0 

y-s 

0 

0 

0 

-2 

-4 

-2 

0 

2 

4 

2 

0 

0 

0 

W(y-s) 

0 

0 

0 

4 

4.72 

0 

0 4 

.72 

4 

0 

0 

0 

0 

I(w)(y-s) 

« 17.44 
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CM 

0 

0 

• 1.93 

4 

1.93 

0 

1.93 

4 

1.93 

0 

0 

0 

0 

r (H ] 2 

a 

6 

15. 

16 

17 

.72 

- 1.1 

a significant error from the correct value 

of 2 


Suppose the function s(x) is a longer duration pulse, but with the 
same edge values. In this case, we observe that if the object is 
homogeneous in the interior, the W term will be zero at those points 
and contribute nothing toward the sum. 




Consider one more experiment, assume the signal is a step edge 



We can now understand intuitively the operation of Fitt's 
predictor.. When an object moves from one frame to the next, a 
signal is produced in the difference image y-s. For objects which are 
homogeneous in their interiors , no information is contained in 
those homogeneous regions, only at the edges. Thus, it is not the 
motion of an object which must be tracked, but the motion of the 
edges of that object. 

With this , we see that 

/w(y-s) de - /(^||>) (y(e) - s(e)) de 

is in fact nothing more than the correlation of the edges of an 
object due to intensity changes with the edges due to motion. 

Thus we can conclude : 

1) Tracking is a gradient-based operation. Only the motion of 
edges is significant. 

2) The motion of the object from one frame to the ndxt must be 



small, not small with respect to the size of the object, but small 

with respect to the distance over which meaningful information can 

* * 

be gleaned from the gradient. That is, motion must be less than 
the edge width. ' 

These conditions result from the antithesis of the continuity 
V 

assumption for images . Images are in fact NOT continuous functions 

» 

of space and intensity, but do contain step discontinuities, and the 
fewer number of grey levels in the digitization process, the worse 
the continuity assumption becomes . 

We now see more clearly why it was necessary for Fennema and 
Thompson [3] to blur their images, for this makes the continuity 
assumption closer to valid, but induces increased uncertainty in 
object location. 

I 

These two conditions above must hold if tracking is to be done 
utilizing only pixel grey value information. However, if some 
image preprocessing has been done, it may be possible to permit 
larger frame to frame displacements. One possibility would be to 
track some feature of the object which is more likely to be continuous 
than its intensity. A typical such measurement might be the vertical - 
cross section. 

The alternative is to admit that digital images are not 
continuous and that classical signal processing techniques simply 
are not appropriate/ particularly if binary images are being tracked. 

We can then deal with the data with that thought in mind. 

A. CONCLUSIONS 

In general, we feel that segmentation is necessary for good 
tracking and conversely, motion information can be a good cue to 
direct a segmentation operator to the appropriate area. Thus, a 
feedback structure is necessary, containing both the "elements of 


segmentation and tracking, with the ability to average out the 
background over time and to extract trackable features from the 
region of interest. 


V 
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5 . APPENDIX 

Derivation of the Fitts' tracker from a first order 
Taylor's series expansion. 

The measured signal y(x) is assumed to have the form 
^ y(x) - s(x-A) + n(x) 

where the expected signal is s(x) and n(x) is noise. For 
purposes of this derivation, we will ignore the noise 'term. 
Then expanding y(x)<- s(x-A) in a Taylor's series we get 


y(x) - s (x) - —A +(— ) 2 A 2 - ... 


6x 


Ax 


s (x) - j^A, or 


( 1 ) 


6s 


A = stx)-y(x) 


We can make this eguation independent of x by integrating 
over the non-zero domain of x. However, since 


i& ax 


s (b) -s(a) 


may be expected to be zero, we must first multiply both 
sides of eg (1) by — (assuring a non zero integral) , and 
then integrate 


b , b 

A /(£~) 2 dx « / (s(x)-y(x)) dx 

a c a 6X 


A ■ i £ — (s(y)-y(x)) dx? where c 
c / x 


/ ( 6x >* dx- 
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Introduction 


The last thing it would seem that the image processing 
community needs is yet another way to compute image gradients. 

The subject would seem to have been exhaused, since it is dis- 
cussed %n numerous textbooks , (2 , 3, 5) (just to list a few) and 
literally hundreds of papers 'we will not list them) (1) provides 
one survey) yet here we are, about to report on another gradient 

operator and worse yet, an operator which is not computationally 

•* 

efficient 1 There is only one thing which motivated us to develop 
this operator, an intense need for accuracy. This is because we 
are modeling frame to‘ frame motion, and using in some cases, the 
value of the gradient to predict the Intensity function in the 
next frame, consequently, we could not be satisfied with an edge 
detector, or some vector in^the correct direction and proportional 
to the magnitude. For example, to represent an image by its 
Taylor's series. 


I(x,y)=I(x 0 ,y 0 ) + |£< x 0 ,y 0 ) <x-x G ) + |§fV y o ) ^o* + .. 


requires that the partial derivative terms be evaluated as closely 
as possible to the correct value, not only in direction, but also 
in magnitude. 


The image processing literature is filled with techniques 
for computing gradients of pictures. There are linear gradients, 
mask gradients, statistical gradients, and many others. Virtually 
all of these return a number which is an indication of the di- 
rection and magnitude of the gradient vector. 

Th simple mask 


f (i,j)-f (i-1,j) 
s f 

yields — at point i,j, however, 

o X 

noise, and operators which cover 
they tend to filter noise. 


such operators are sensitive to 
a larger area are desirable since 


Sobel (2,P271) proposes finding |~ at (i,j) by computing 
*<i+1#j) - f(i-1,j). 

Where f (i,j) is an averaged version of f(i# j) computed by taking 
a weighted average of pixels about (i,j) in the vertical (y) di- 
rection, ^his gradient has good smoothing properties# and there- 
fore is not so sensitive to noise# however some observations need 
to be made: 

1) The Sobel operator actually returns twice the gradient, since 
it is taken over a space of two pixels. 

2) The operator achieves smoothing by throwing away information# 
the center pixel is ignored. 


Some authors EPrewitt# RosenfeldD point out that one way 
to find the gradient is to fit a curve to the set of points# and 
then to differentiate that curve. In this paper# we propose to 
fit a plane to a 3x3 set of points# and the slope of that plane 
will be the gradient. 

3f a f 

We will compute -r- and -r— separately# and in fact reduce 

oX dy 

the problem of fitting a plane to two line fitting problems. 

For purposes of clarity# the following explanation con- 
df 

siders only -rrr. The argument for the y axis is symmetric. 


Definition: f(i#j) is the value of intensity at point i#j# 

where i corresponds to the x coordinant and j to the 
y coordinant. 

Definition: f (i# j) £ (f (i# j-1 )+2f (i# j)+f (i# j+1 )/4 is the 

weighted average (as in the Sobel operator) of the 
intensity at a point# averaging being done only in 
the y direction. 


Thus# in a 3x3 neighborhood# we can compute three hori- 
zontal intensity values (for notational convenience, we will 
rename these variables) f(i-1#j)-I a # f(i#j)-I b # and f(i+1#j)=I c - 


9 


9 


4 


i 

We now wish to fit a straight line to these three 
points. The slope of that line will be the gradient at i,j. To 
avoid the coordlnant-dependent degeneracy problems pointed out 
by Duda and Hart C2J, we will use eigenvector line fitting. 

We will consider each point as a vector spanning the 
space <x,f(x)> (having eliminated the y coordinant for the time 
being . 



example 



Then the vectors a, b , and c can be defined: 



9 


Let the mean vector m T ^ <m x ,mj> 
where m x «x b and m I *(I a +I b +X c ) /3. The best fitting line will pass 
through this point. 

We now transform the coord inant system to put the 
origin at m by subtracting m from each point. 

V T 

v a " 

v b “ < ° 

»* > 

v c * < (x c“ x b ) » (I c" ra I ) > 


Now, we compute the scatter matrix of the three points, 
(define the scalar m * m^) . 


S = Z v, v! = 
i-i 1 1 


< x a* x b> 
(I a - m ) 


[ (x a- x b> (I a- m) ] + 


r » 1 [o 

L <!>,-"» J 


'V 1 "'] 


( x c- x b»1 [ <x o _x b> (I c' m) J 


L( 


x c- x b>' 
I -m) . 


We assume unit spacing on the x axis, so (x -x b )«1 and 
x b «x a =-1 , and S simplifies to 


S = 



(I c -m)-(I a -m) 


L (I c -m)-(I a -m) 


(I a -m) 2 +(I b -m) 2 +(I c -m) 2 J 


Define t; * (I -m) 2 +(I.-m) 2 +(I -m) 2 and A = I -I , and simplify 

a D C C 3 

S some more yielding 





In the example above A * I -I « 10-4 
V c a 


_ _ 10+8+4 

m - g 


■ 7.33, and C * 18.665. 


6 , 


The line which best fits the data will be in the di- 
rection of the principal eigenvector of S. Thus we must first 
find the principal eigenvalue of S. The characteristic equation 
is derived from 


det (S-XI) 


(2-X).„ 


A 


A U-X) 


* X 2 - (2+C) X + ( 2 C — A 2 ) . 


Setting det (S-XI) = 0 yields 

X = 1 + C/2 + ( v (2+C) 2 - 4 (2C-A 2 ) /2 

- 1 + C/2 + 1/2 y/ (C“2) 2 + 4A 2 . 

Since we wish the largest eigenvalue, we will choose the 
positive sign. 

Now solving 



= o for e 1 and e 2 will yield 


the principal eigenvector 


The units we shall use for the gradient will be inten 
sity units per pixel* Thus# we will define the horizontal axis 
to be measured in units of pixels. Therefore we can assume 


x, - x. « -1 . 
a d 

v In this case, let < 22=1 then e^=A/(X-2). So the prin- 
cipal eigenvector is C(A/(X-2)) 13 and the best linear approxi- 

mation to the 3 points is a line passing through m in the 
direction c(A/(X-2)) 13. The slope of that line is 


(1) slope * 

f 

which is the gradient in the x direction. 

In the example: •*, 

= 1+ U|i§5 + (2+1 8 . 665) 2 - 4(2(18. 665)-36) = 20.602. 

31 _ 20.6 - 2 _ * , 

9x 6 ” J *' * 

We should observe at this point, that it is not 
necessary to assume the sampling rate (x a -x b ) is equal to one. 

If instead, the sampling rate is x^x^ = -k, x b -x c = 
for a positive real number k the solution yields: 

(for I c = I a ) 


(2) slope = 


X„ v -2k‘ 
max 

kA 


where X 


2k 2 +E+V(2k 2 -S ) 2 +4k 2 A 2 " 


max 


The solution (1) is degenerate in the case I c = I a , since the 
denominator then goes to zero. This case is best handled by 
explicity testing for I = I . In that case, one must then 
test if C<2. If so, the gradient is zero, otherwise infinite. 


Conclusion 


This paper has derived a mechanism for computing image 
gradients from 3x3 neigborhoods. The method is based on fitting 
a straight line to the 3 points corresponding to a weighted 
average of the 9 points in the x (or y) direction. The method 
is optiriial in the sense that it minimizes the summed squared 
perpendicular distance from the observed points to the line. It 
is not efficient computationally/ since it requires calculation 
of a square root/ but does provide highly accurate estimates of 
the image gradient in the neighborhood. 
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ABSTRACT 


Region growing is often given as a classical exaaple of the nse of 
recursive control structures in iaage processing. Recursive control 

structures', however, are soaewhat awkward to build in hardware, where 

» 

the intent is to segnent an iaage at raster scan rates. This paper 
describes a hardware structure capable of perforning region labeling 
iteratively, at scan rates. Every pixel is individually labeled with an 
identifier signifying to which region it belongs. 

t 

The difficulties which often justify recursion ("0" and "N M shaped 

regions, etc.) are handled by aaintaining an equivalence table in 

« 

hardware, transparent to the coaputer, which reads the labeled pixels. 
The uechanisa for updating the region nap is explained in detail. 

Actual ieplenentation of such a systea would require a content 
addressable read/write aenory. Such aeaories are now feasible, using 
existing LSI fabrication techniques. 




1. Introduction 


One central problee arising in scene analysis is the partitioning 
of an inage into a set of aeaningful regions. These regions are conposed 
of all pixels that have sinilar attributes (such as grey level) and that 
aro connected to each other. One of the nost powerful approaches to this 
task is "region growing" <3>. This technique is initiated by choosing a 
pixel which neets the criteria for inclusion in a region. It then 
proceeds by examining all adjacent neighbors of the pixel and 
incorporating into the region all neighbors neeting an acceptance 

criterion. This acceptance criterion is based upon the siailarity 

* <« 

between the pixel and the neighbor in question. Typical aeasures of 

I 

siailarity include the nagnitude of the neighboring pixel's grey level 
or the relative contrast between the pixel and its neighbor under 
consideration for inclusion in the region. This process is repeated 
recursively for all newly accepted pixels until no new pixels can be 
added to the region. Then a new region is "grown" around a pixel which 
has not been assigned to a region. When all pixels have been assigned to 
a region, the algoritha terninates. Since the region growing technique 
always results in closed regions, this techniques is soaetines 
preferable to other techniques which are based on edge detection or line 
fitting. 

Nunerous variations and applications of the basic region-growing 

j * 

technigue have been proposed in the past. A critical coaponent of the 
region growing technique is the selection of appropriate acceptance 
criteria. Por exaaple, if the acceptance criteria are based on the 
thresholding of the contrast between a pixel and a neighboring candidate 
pixel, situations can arise in which a final region does not neet the 


original acceptance criteria <*>• 


Brice end Pennene <1> overcaae this problen with a region aerging 

procedure, in inage is initially partitioned into a large nuaber of 

V 

regions each of onifora grey level. A pair of siaple heuristics is then 

t 

used to aerge siailar neighboring regions. These heuristics are based on 
the strength of boundaries separating regions and the rate at which the 
boundaries of aerged regions increase. Yakiaovsky and Peldaan <2> 

described a siailar procedure in which the region aerging criteria are 

#• 

founded on Bayesian decision theory and heuristic techaigues. 

Nilgraa et. al. <11> have described the super-slice algoritha for 
region extraction based on edge detection and variable thresholding. 
First, a thinned edge picture of an iaage is obtained. Next, a range of 
thresholds is chosen, and the regions resulting fron each threshold are 
coaputed. The threshold resulting in the highest percentage of border 
points which coincide with the thinned edge points is chosen to define 
the regions. This aethod bears siailarity to the work of Krakauer <8>. 

Although region growing has proved to be an integral part of scene 
analysis, its use can guickly becoae computationally prohibitive, 
particularly for high resolution inages. This has pronpted soae 
researchers to consider alternative nethods of region partitioning. 

2. An Algoritha for Region Partitioning 

The algoritha for region partitioning presented in this section is 
a region growing technique based on the concept of equivalence 
relationships between the pixels of an iaage. Two pixels a and b are 


defined to be eguivlaent (designated R(a,b)) if they belong to the sane 
region of an iaage. This relationship can be shown to be reflexive 
(F(a,a)), synnetric (R (a, b) <*>R (b, a) ) , and transitive 

(R(a,b)AR(b,c)*>R(a,c)) . 

V 

The transitive property enables all pixels in a region to be 
* determined by considering only local adjacency properties. In this 

algorithm, each pixel will be conpared first with the pixel to its left 
and then with the pixel -above itself in a lef t-to-right, top-to- bottom 
raster scan fashon. The assignment of a region label to a pixel results 
from this conparison operation. Figure 1 denonstrates the situation tha 
can arise as a result of this conparison. Pixels in a simple binary 

i 

inage are being labeled in raster scan order. The region labeling 
proceeds in a straightforward manner until the eguivlaeace relation 
R(1,2) is discovered at the pixel designated by the question mark. 

The system proposed in this paper employs hardware to assign regio 
labels to pixels and to maintain a table of equivalence relationships. 
Figure 2 shows that this hardware resides between the iaage memory and 
host computer. Functionally, this hardware is transparent to the host 
computer. For the example of Figure 1, all pixels will be perceived by 
the host coaputer as belonging to region 1 (the lower numbered region 
label takes precedence in an equivalence relationship) . 

The operation of the hardware is described in the flowchart of 
figure 3. In order to understand this flowchart, the following notation 
is introduced: 


I(x,y) is the grey scale value of the (x,y) 
pixel in the inage nenory. 

B(x f yi is the region label nunber corresponding 

to the (x # y) pixel in the inage aeaory. 

K (i) is the contents of the i-th eleaent in the 
equivalence aeaory. This aeaory ia a 
content-addressable aeaory. 
iaplies the sequence: 
v 1) read K(j) 

2) search K using i as a search key 
(i.e. deteraine all 1 such that K(l)»i 

3) write K(j) to all positive responders 
of the search. 

T is a threshold 

P is the highest naabered region label 


The algor it ha is illustrated in Figure * for as arbitrary region 
adapted froa niigrae et, al. < 11 >. 

As the region partitioning proceeds in real-tine (i. e. 
synchronously with the raster scan), tvo activities oust be perforaed. 

First, the H aeaory east be loaded with the region label nuaber of each 

* 

pixel under consideration, and second, the K aeaory aust be updated with 

all equivalence relationships discovered. For exaaple, if region 4 is 

actually identical to region 2, then both K(2) and K(4) will contain 2 

(the lover nuabered region label takes precedence) . Hence, when the host 

coaputer interrogates pixel (x,y) of the n aeaory, the 

» 

interface/processor interprets H(x,y) in terns of the K aeaory and 
returns K(H(x,y)) to the coaputer. For exaaple, pixel (10,10) in Figure 
n would be returned as belonging to region 1 since K(H(10,10))*K(4)*1. 

The difficulty generally encountered in this type of procedure is 
the problea of chaining. That is, if R(2,4) and R(3,4) have been 
deterained, then R(2, 3) aust also be deduced. However, to reguire that 
the coaputer search out all such possiblilities after the iaage has been 
processed defeats the original objective of perforaing region 
partitioning during the scan tiae. Chaining is avoided in this algoritha 
by ensuring that K (2) *K (3) (4) =2. However, this aerely transfers the 
chaining problea to the scanning and labeling process. Block 4 of the 
algoritha flowchart shown in Figure 3 resolves the chaining problea. 
Vhenever an equivalence relationship is detected, all locations in the K 
aeaory containing the larger region label nuaber are loaded with the 
saaller region label nuaber. Bhile the execution of this step in 
real-tiae is patently absurd for conventional raadoa access aeaories, it 


ill within the capability of the conten t-addressable aeaories discussed 
in the next section. 

3. An Architecture for Ispleaentation 

V 

# 

The architecture proposed to isplesent the algorithm of section 2 

is shown in Figure 2. .This hardware is intended as a special purpose 

« 

processor for an existing conputer-based inage processing systea. Mori 

et. al. <9> have also described a special purpose region labeling nodule 

* 

Used with the Toshiba Pattern Inforaation Cognitive Systea. However, 
their systea was not intended to operate at real-tine (i.e. video) 
rates. 

I 

The architecture of Figure 2 contains four aajor conponents: iaage 
aenory (I), region label ueaory (M) , equivalence uesory (K) f and an 
interface/ processor. The grey scale values of the isage reside in the 
isage aenory. Typically, the I aenory would contain 512x512 bytes. The 
tegion labels assigned to individual pixels are contained in the region 
label aenory. However, the contents of the H aenory also include all 
interaediate region labels for which equivalence labels were deteraiaed. 
Therefore the contents of the H acaory aust be interpreted in teras of 
the contents of the equivalence aenory. The size of the M aenory is 
directly related to the bit length required to represent the region 
label (including interaediate region labels) associated with each pixel. 
This problea is further discussed in section 4. 

The I aenory and the M aenory are both conventional randoa access 
aeaories. However, the equivalence aenory is a content-addressable 
aenory. A content-addressable aenory has the property that neaory cells 


cid be accessed or (in this application) loaded by their contents 
<5,6,7>. In this application, the content-addressable capablilites of 
the K nenory are used as follows: The K aeaory is considered to have a 
data bus into it, an address bus, and several control wires, (see Figure 
5). In conventional read or write operations, the address is placed on 
the address bus, the A control line set to zero, and the data will 
appear on the data bus synchronously with the 61 clock line. In this 
node, it performs as a conventional RAH. In the associative, node (A*1), 
the data bus is interrogated during 61, and all nenory locations in K 
are coapared with the contents of the data bus at that tine. For each 
aeaory location, there is a flag flip flop (F) which will be set or 
Cleared according to whether or not the contents of that cell natched 

I 

the contents of the bus. Then, during 62 tine, the data which is then on 
the data bus (presunably the new equivalence) will be read into all 
cells whose flags are set. Thus, the operation K*(i)*J can be perforned 
in two cycles of a fast clock. 

High cost and technical linitations on the size of these neaories 
have traditionally hindered their widespread acceptance and use. 

Hardware implementations of content-addressable nenories have enployed a 
wide variety of perforaance. However, the developnent of VLSI and VHSI 
technologies should stiaulate the use of such nenories. 

Critical design specifications for the K nenory are the access 
speed, aenory size, and the ability to write an arbitrary number of 
nenory locations sinultaneously. Host techniques for building 
content-addressablenenories emphasize one of these paraneters at the 
expense of the other two. For exaaple, the nulti-dinensional access 
(HD A) nenory of the STABAN conputer <10>, with its bit-slice structure. 


achieves the last two specifications at the expense of aeaory speed. Th< 
parameter eost crucial to the development of a satisfactory aeaory, and 
hence a systen capable of operating in real-tiae, is the aeaory size. A 
near real-tiae systea will result if the aeaory size necessitates a 
coaproaise in the access speed. The aeaory size is discussed farther in 
Section • where sinulations involving real iaages are investigated. 

!' 

The final conponent of the proposed architecture is the 
interface/processor. The priaary purpose of this unit is to execute the 
algoritha described in Section 2 and flowcharted in Figure 3. 
Additionally, it nust be capable of 

1) processing the video signal input into grey scale values for 

t 

storage in the 1 aeaory, and 

2) interpreting the n aeaory in teras of the K aeaory. 

The ability to process iaages in real tiae will uadoubtedly 
necessitate a design based on special purpose, high speed, bipolar 
coaponents. However, if near real-tiae performance is satisfactory, 
currently available bit-slice aicroprograaaable aicroprocessors should 
prove adeguate. 

4. Siaulation 

The algoritha was applied to a 128x128 television iaage of aoderat< 
coaplexity. Three cases were evaluated: 

1) 11=12, 4 connectedness 

2) ABS (11-12) <3, 4 connectedness 

3) IBS (11-12) <3, 


8 connectedness 


In case 1), for two pixels to be considered in the sane region, 
they bad to have identical intensities, whereas in cases 2) and 3), two 
pixels were considered in the sane region if their grey waives differed 

by less than 3 (in a 6 bit inage) . 

V 

In case 3, the center pixel was coapared, not only to the pixels 
tbove and to the left,., bnt in addition, to the pixel above and to the 
right (northeast) . This is not true 8-connectedness, since the upper 
left pixel was not tested, but it deaonstrates the redaction in 
conplexity which results fron this ratner sinple change. 

Three paraaeters are of interest. 

I 

1) the nunber of elenental regions (those whose labels are stored in H ) , 
since this affects the word width of N and K, and the length of K 

2) the nunber of eguivalence relations detected, since that indicates 
aaount of data reduction w r ,.ich occurs, and 

3) the nunber of regions perceived by the coaputer, since that deteraine.' 

the anount of further processing which the coaputer nust do before 
useful infornation can be gleaned fron the inage. 


The results are sunnarixed below 


Case 1) 1589 elemental regions (requires 128x128x11 H and 

2048x11 bit R) 

42 equivalences 
1554 perceived regions 

C'se 2) 392 eleaeutal regions (requires a 128x128x9 bit H and 

V 512x9 bit K) 

315 eguivaiences 

291 perceived regions (a significant reduction) 

Case 3) 318 elenental regions 


412 equivalences 
241 perceived regions 


The simulation required (I seconds in PLI on an Amdahl computer. 

Figure 6 shows the contents of the H nenory for a saall area of the 
isage, and Figure 7 shows the sane region as perceived by the coaputer. 
It is interesting to note that nost of the snail regions occur because 
edges have widths greater than one pirel, a fact that is reasonably easy 
to detect and filter o.ut (in this inage) . 

5. Conclusion 

In this paper we have addressed the issue of perforning inage 

■ »* 

analysis operations in real tine on television scanned data. He have 

» 

shown that it is possible to design hardware which can perforn the 
operation of region growing in this way. The concept of using 
equivalence relations to partition an input set is fundanental to the 
algorithm. And the use of content-addressable read/write nenories is 
essential to the inplenentation of such equivalence relation processing 
in real tine. This raises the question of whether or not such 
associative lookup structures cannot also be used for other 
traditionally recursive conputations. 
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Figure 7 

Contents of the region label memory as it 
appears to the computer, having been translated by 
the K mapping 
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AMTAACT i of a) action detection, b) attantioo directing, and c) 

cognition. For practical raal tin* application* thaaa ’ 
Computer tracking of Boring object* using television process should run in parallel, 

imagery is discussed. Problems which are unique to 

this field are described, including caoera motion, Under the category of trackers, vt distinguish two 

occlusion handling, and difficulties with correlations . types of systems, background trackers and target 
These problems are described in the context of seve- trackers, background trackers identify same feature in 

ral solution approaches mentioned in the literature, a background (a.g., a bridge), and track that feature, 

A field-operational system is described which demon- even though the camera may move, background trackers 

strates the ability to deal with these problems with have found applications such as remotely -piloted vehic- 

some limitations. Finally, future direction for * lea, "smart bombs", which must home in on a target, and 

research in this area are proposed. u is:,* LA* D6 AT -type satellites which must identify and track 

ground control points to increase their pointing accu- 
racy. In all these applications, the intensity function 
can usually be considered to be continuous and diffe- 
rent i able at all points, with non-tero derivatives at 
most points. (As we will discuss in a moment, these 
assumptions are important to the design of the tracker). 


i. iirruoDucTio* 

It was only a few years ago that even the simplest 
image processing operations strained state-of-the-art 
computers to their utmost limits. A 126x126 array took 
a lot of memory, and even the simplest operations re- 
quired enormous amounts of time. Today, with faster 
computers and cheaper swmory, image operations still 
take a lot of time, but it is no longer unthinkable 
to process 612x51 2 images. So now, with the tech- 
nology catching up, imaging devices are expected to 
become an increasingly important type of sensor for 
closed-loop controllers in different kinds of "in- 
telligent systems' 1 such as occur in robotics, missile 
guidance, and automatic tracking of moving objects. 
These applications require real time processing of 
sequences of images, and once again the demands of ima- 
ge processing strain the technology to its limits. 

We define Computer Analysis of Time-Varying Images 
(CATV1) to mean the processing of a time sequence of 
images in order to extract some property which changes 
in time. There are two types of uses for motion infor- 
mation : 

1) Where the motion information itself is what is im- 
portant. Trackers typify such systems. 

2) Where the motion information is used to provide 

additional features to scene analysis systems. 


A target tracker may have to deal with an entirely 
different situation, for example, an airplane against 
a daytime sky. In such applications, the sensor is 
often saturated, resulting in an image with high con- 
trast and low dynamic range. The region(s) in the 
image which describe the target are then homogeneous 
in intensity, and motion information can be extracted 
only at the edges of ths regions. 

Gilbert C9 3 extends our definition of trackers 
slightly to distinguish between contrast trackers, 
which threshold the intensity, edge trackers, which 
threshold the magnitude of the gradient, and corre- 
lation trackers, which correlate shapes, based on 
intensity. 

Under our category 2) above, we can make use of 
motion information either for image segmentation or 
for target classification. Segmentation implies 
grouping of similar pixels. For example, velocity in- 
formation could be used as s similarity measure to di- 
rect segmentation. In addition to grouping of pixels 
of a similar velocity or acceleration together and 
thus aiding segmentation, that same velocity or accele- 
ration data can be used to provide features for a 
classifier 

The ares of CATVI can be subdivided in many ways, 
ccording to objectives of the system, as we have done, 
or by ths methods used, as Nagel C 1 5 3 hat done. In 
this survey paper, Nagel points out that the basic 
problem of finding relationships between images in a 
sequence can be broken down into several steps: 


Since in order to classify a target one must first 
locate and track it, systems whose primary goal is of 
type 2) generally incorporate components of type 1), 
Such systems can also be decomposed (as Martin and 
Aggrawal C 1 3 3 did) into components which have the tasks 


1) Derive a description (or descriptions) of the image 
substructure , 

2) Establish s (dis-) similarity function of these 
descriptors , 
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3) Uit Uii function to tiUbllib eoapttifiiitf or li- 

compat ibllity b«U*#r subst rurla/»i , and 

O A« i rtiull of tbe comparison, 4ticrib« th# pert i- * 

nent or poets of the 1 * 04 * sequence. 

Rt|*l then foot on to survey tbo Xltorsturo accor- 
ding to this schema. 

Our purpose is ibis poper is net to provlae s sure#/ 
of tbo literature, but to discuss tracking . The rosdor 
Is directed to tbo aforementioned survey by Mogel, sod 
to tbe survey by Kurt in end Aggnrvsl HJ) for s thorough 
litsrsture review, bo would sent ion one interesting 
item et this point, however. As Hertm u.i Ag#srv%. 
point out • the fundamental problem in CATV2 is thr 
problem of occlusioo. This observetioo brings us to 
discuss wbst we perceive es tbe principal reseerch 
problems is this eree. 


2. DIFFICULT SITUATIONS FOR CATV I 5 1ST DO 

Although this is e very now eree (tho military hes 
been in tbe business s bit loader then tbe Univdrsity 
community), there hove been nonetheless s considerable 
number of papers published, and it is instructive to 
note the simplifying assumptions which people bsve made 
in order to develop systems which word. The most cosmon 
assumptions are: ^ 

1 ) The comers does not move 

2) The target is never, or never totally, occluded 

3) Image gradients are well defined 

Itea 3) is often an implicit assumption and seldom 
stated outright in this form. We will discuss these 
three items briefly. 


2.1 Cwr» KQtjcn 

The most straightfo* eord neons for tracking mo- 
ving objects is to wc vitb difference pictures. 

This entails computing the sbsolute value of the 
difference between two successive frsmes, on s pixei- 
by-pixel bssis. Then, with s stationery background, 
the backgrout.d pixels subtrset away to xero, and the 
moving target stands out cleerly. 

But consider now a TV tracker with a servo on the 
pen and tilt of the TV camera. If the tracker works • 
perfectly, the target will be in the center of the 
screen st all times. Freae tc freme differencing then 
subtracts out, not the background, which changes with 
earners angle, but the target! If we then add s servo 
controlled toon lens to the system, the problems be- 
come really interesting! 

Of course, if all these effects sre precisely known, 
they can be computationally taken into account. However, 
it is doubtful whether these effects will ever be so 
precisely known that difference methods on s pixel by 
pixel basis slone will work in real situations. Any 
sophisticated algorithm must therefore ultimately be 
tested against real world conditions. Robustness, the 
ability to adapt to unexpected circumstances , is s ma- 
jor requirement. 


2.2 Occlusion 

The target may occasionally be either partially or 
totally occluded. In the esse of partial occlusion, s 
number of approaches ore possible, including subtrac- 
ting the occlusion to find the non-oceluded portion(s) 


of the target, and then matching thoae parts to the 
model of the target, to get on updated target location. 
Thle matching problem can be quite difficult In th# 

general case. 

When the target disappears behind an occlusion, 
tra gfciag is, of course, impossible , One clue for band- 
ying this situation may be derived by observing people. 
Confronted with a moving target which flys behind an 
occlusion, people tend to extrapolate the velocity of 
the target and prediet its point of reemergeor# from 
behind tbe occlusion. They then direct their atten- 
tion to that point. A backup mechanism ia the form of 
periferal vision seems to be provided in case the tar- 
get does not emerge es predicted. Thus, to emulate tbe 
occlusion handling capabilities of peopls, s computer 
system neede prediction, attention directing, and 
"peripheral vision** coupled with backup capabilities. 

Tet another class of problems occurs when the occlu- 
sion is itself a moving object. In this case, same 
classification is necessary to distinguish the two ob- 
jects when they separate. Among the more recent work 
along these lines is that of Martin and Aggarval CU], 
ex* | . Roach and Aggarv* 0 edges 

of objects are segmented into s set of approximately 
straight lines and circular arcs. Whils this sejpen- 
tstion again appears artificial or "man-made", the 
idea behind it, i.s., matching edge segments of par- 
tially occluded objects using motion information to 
"analyte an apparent object into its constituent ob- 
jects**- represents an important step in the direction 
we think occlusion analysis should go. In section 3* 
we discuss s very simple but affective form of occlu- 
sion handling in on operating tracker system. 

2.3 TH* Protlf wit* E<U*l7jr«troc«.»if>J 

Digital images, particularly images of man-mad* ob- 
jects, tend to have areas which ore homogeneous in 
intensity. If one places s small window or local opera- 
tor about a pixel in the center of a homogeneous re- 
gion, and then looks st the. same ares in the next 
frame, no change will be observed unless an edge moves 
through that window. This situation is most extreme in 
the case of binary images and contrast trackers. Mo- 
tion information can be derived, not from the motion 
of regions or objects, but only from the motion of 
edges. 

We should point out hers that our use of the word 
"edge" implies on ares where there is s significant 
rate of change of intensity with respect to distance 
in picture coordinate. Edges and object boundaries 
are not synonymous. While object boundaries usually 
give rise to edges, the existence of an edge does not 
necessarily imply a boundary, since objects may have 
internal detail which gives rise to internal edge*. 

We have observed the fact that motion information 
comet from edges occur ing several times in current 
literature; we will point out two examples. 

First, consider the velocity estimator of Fennerta 
and Thompson C53. They point out that motion can be 
represented by s velocity vector 


and that if ds is sn incremental translation of the 
surface, and di an incremental variation in intensity 
due to this mot ion, then 
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nU bicbfroufti tricbui Un rootiiai*trifbln|V fitti 
bit ilforilht by matched Mtiiini tbforj, 


it U IHCftHill to not* that bit tracker can 

K^*laee tbt frame -te- frame intensity change at a poiat . # . aleq be fro# firtt or der r Taylor ’* atritt ta- 

ll known , and tbt iiaditbt Computable tbto it should 1 Sumptions Tbit derlvatioall |lvtn in tbt appendix- 
be possible to find tbt velocity of each pixel. V ^ ^ v V v 

" ' Tbt point of th#t# |« Mtapltt drawn from tht U- 

Auothtr pap tr (191* ehowe that Fonntoa and Thom* » • Itraturt la tiflBlF ibit| notion information aiiita 

.on caa bt derived from a f irtt ordtr truncat ion J( locally) only in tht adfti , and tbit fact nuat bt 

of tbt Taylor's aeries tipaotioo of tbt intensity funr |b#pt in aind when designing a CATVJ ayttto. Since it , 

lion. • ’ \ 1 • clear that act n n UlMmill it to lntnntlcly in- ; 

■KZpolttd with tbt etructrurs of iattnaity adjex, oat * 
.might than aab if it dott aot make ttntt to bandit ibt ! 
extraction of edges at a ttparatt oparatioa. Further- j 
mo rt art tbara not othtr, timilar oparationa which 
could bt handltd *t a low, preprocessing levels. Thtrt 
it a grant dtal of evidence that organic ayatama per- 
l f orm aucb praprocaaaiag function*. 


_ . For thia mat bod to work , the* gradiaat at tbt piial 
baiag examined mutt bt tht aamt in both frames, wbicb 
iapiiaa dJ/dt ouat bo wary amall or taro, fannaaa and 
Thampaoo accaopiiab thia by blurring tht adgaa to la- 
aura that tba width of tba adga it graatar than tba 
frame to fram* diaplacamant , to that, on tba average, 
0 dot* not wary from fram* to frame. 
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Analysis of t lme-vary ing images looba at first libt 
radar signal procaaaing, tinea tba objective it to pick 
out a known signal from torn* background (not at). How 
star, dua to tbt 2-dim*neiooel naturt of the mignel , • 

tbt probability of bomogtntoua arena, and tbt likali- 
bood of occluaiooa, ciaaaienl correlation techniques 
baaed on iotanaity alone ar* not likely to ba success- onB 
ful. At Draacblar and Hagai C33 point out, 


It might ba arguad that croescorrelatioo batvben 
objact candidates extracted from two consecutive 
framaa should allow tracking t|a imaga of a . 
moving objact... Tbara art aaveral weak points 

to aucb an approach... Unlaat a tight aubaaction . ^lational tbiftt of theae linaa. 

■ _ . . 1 . . • . w l • w . ; , • » . ■ . . .» «• 

imaga of a moving objact can bt dt- 


For inalanct, Gaardntr (63 haa pointed out that tbt 
human viaual ayatam heavily depend* on an intrlaaic | 
feedback ayatam involving aya jumps even in tba fit- | 
at ion cast. It it indeed stay to show that whan tba 
ay* it filed oa a certain objact without movement , the 
image of tba objact blura dua to adaptation. However, 
amall aya jumpe product refreshed images resulting in 
.maw information only at the edges. Thtrt art othtr 
hint* that in mammalian visual systems there it a lot 
of feature and adga enhancement . From Hubei and 
Wieaela'a work MU, vt know that in the cat's visual 
rwprcortei there art neurone functioning as adga or line 
detectora, with special sensitivity to orientation. 

Soma of that* functions are even independent of trena- 
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around the image 

t trained fairly accurately. • . the croaacorra- 
latioo of such subsections from consecutive 
frames will depend heavily on tba structure 
of tba atat ionary image components m tba se- 
lected aubaaction. Tracking of the image of a 
moving objact tbua tends to become unreliable. 

On might attempt to circumvent this difficulty 
by croaacorralating difference picture! ob- 
tained with reapact to a rafaranca fraae. Thia 
will only succeed if the leaf a of aiactly one 
moving objact dominates the difference picture- 
an assumption which might be difficult to check 
automatically during tbit stage of tba analysis. 


For theae reasons, wa feel it ia necessary in ge- 
neral that processing of images of moving objects ba 
based on "matching" of features other than simple inten- 
sity functions. However, with the correct assumptions, 
the concepts of correlation and matched filtering can 
ba used, although aucb a aystaa will inevitably encoun- 
ter this same problem with edges. 

One auch system, developed by Fitta C63 encounters 
the problem with edges in a slightly different way. Ha 
determines that the displacement of a point 6 X can be 
computed with only two ooe-dimenaional correlations , by 
using 


i* • i 1 - u •’ )_,U): 44 


dg it the estimate of 6 X , y(i) is the measured in- 
tensity function, (assumed to ba y(x)«e(x-6 x )en(x) ) , 
and S it the known (tracked) signal. Again, on* obser- 
vea that information exists only in the edges, for to 
have an non-saro integrand, the tracked signal must 
have a non-taro derivative at the same point that it 
has s difference dua to motion. It it tbua more suitable 


Thera have bean different artificial approaches to 
preprocessing pictures for feature enhancement. Such 
preprocessing serves one of two basic purposes : to re- 
i duct the dimensionality of the feature space, or to 
increase the orthogonality of the feature vectors M23, 
both of which potentially lead to batter performance 
of a matcher. One could, for example, preproceaa to 
extract a segment of the target contour, and than match 
that segment, 
i 1 thin T 1 1*1 

Another form of preprocessing results from the use 
of image transforms. The general area of transforms 
has been studied extensively, to the point that euch 
transforms are textbook material Cl, 173, but only 
some work haa been done on transforms in the context 
of CATV! C2 , l», 18, 203, particularly with reepect to 
transforms of segments of an image with the intention 
of removing or identifying translational effects. For 
example, at flret glance, the properties of the Fourier] 
transform seem very striking, eince the amplitude 
response is insensitive to translational shifts, while 
the phase response immediately shows up the amount of | 
auch ehlfta. But in reality there never it a whole flc- 
ture which «■ completely shifted, but only a small part 
in it which le not identified a-prlorl. Thus the 
Fourier transform simply has a different appearance, 
without a dlecernable correspondence between the ehift 
of the target and the response of the transform. Con- 
sequently , one must also temper hie enthualasa when 
considering "obvious" applications of transforms. 

. i 

Ve have mentioned some of the difficulties with 
analysis of time varying Images and same of the pro- 
blems which one encounters when attempting to apply 
conventional tools to thoee problems. We will now des- 
cribe one operational system which functions reasonably 
well in thia environment. 
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^ ^2b this mtloa , w will 4#icrib# 't » *.T tracker rmgt ioiieOordi amt so ^ a*«a ^ ' processing of* the 

Wtt«0 Which is durromtly operational at UFYU UO,. a. la fra— k, >o r » ^ - y t , » fitnaa from Yr 

rta a contrast tracker, 'In which the came ra >ar» and tilt determination of concurrent computation of window and 

are oorvoed automatically to keep the target cantered vVjpttraaa (max, ala) N ^ "varners position Tor fraae 

>a the field af vie^. It can aeccmodate oc el as ion* , ^ Jfor fT a»# k - O-. ,Sk*1 


iwitb eoaa limi tat u*>s ) , and rotations and site changes 
in the target. 


The s/sten la built around a critical dssign phil©-^|- 
aopror • it it acre important to be robust than to b# ^'n 
accurate. That is, aaai' inaccuracies in computed tar I' 
get location, velocity, and description of target shape 
can be tolerated, but the tracker must not ever loee the 
target. 

Thin section describee tba DFVLT level 1 tracker, 
which ia operational in the field. Our level 2 tracker 
provide# greatly improved ability to distinguish tar- 
get ahapea, but in currently implemented only in simu- 
lation, and will not be dincueeed here. 


3.1 Tht gMflll flit™ 

Fig. 1 shove a amplified block diagram of the 
tracking eyatea. The video signal is supplied by a , t ions 
black and white gimballed camera and displayed on a 
color monitor. (The color functions of the monitor are 
used to overlay the window location and Vreshold bound- 
aries). At the same time the analog video signal is J r 
cam, vred in the signal processing logic with an ad- 
justable grey value level. Whenever the amplitude of 
the video signal crosses this threshold, either rising 
or falling, the corresponding line mmber and pitel 
number in this line are registered from counters syn- 
chronised with the frame and line sync pulses. At the 
end of the line these values, having been accumulated 
in a fast memory, are transmitted to a process com- 
puter. 

Signal evaluation (from the hardware) occurs only 
within a window which is normally adapted tc the ob- 
ject site automatically by the computer. The computer 
then has the task of calculating the center and ex- 
tensions of the target by the given contour values 
(including window calculation) and servoing the camera 
so as to null the deviation between object center and 
screen center. 


, Table 


Software activity vr.il* fr 
recorded . • vVv 
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The level 1 tracker is characterised by an enor- 
mous reduction of information flow. It uses only the 
first and last threshold values in one line (inside the 
window) and it actually does not process all values of 
the object contour, but only the extrema in the hori- 
sontal (i) and vertical (y) directions. So only four 
values are actually encoded in each frame and used for 
decision aaking. These four extreme may be due to the 
target contour, or they may stem from occlusions ente- 
ring the window. It it the task of the logical decision 
part of the computer program to cancel implausible 
values and replace them by estimated ones. 

3.2 Structure of the camera control loot 

In this section, the closed loop for camera control 
(in x and y) is briefly outlined in its dynamical be- 
havior. An essential implication is the temporal re- 
lation between acene perception processing and rea- 
lisation of camera motion, (taole 1). 


1 With each frame starting pulse, the same operations 4 
*are Initiated, Which (s ahsT^gou* X * rill 
data system. While frame k is recorded from the video 
signal , the computer is processing tbs contour extrema 
of frame k-1. This program ?• concurrently interrupted 
by the transmission of the contour values of the running 
line of frame k. Coupled with this transmission is a 
■ in-max search so thst st ths end of frame k the corre- 
sponding extrema are known. Processing of frame k-1, 
being s background job is also finished at the end of 
frame k. The cam;?uter has by the end of frame k checked 
the supposition target motion for plausibility , correc- 
ted it if necessary, predicted the target position two 
frames in advance, and calculated window coordinants 
and camera po® » fMM fco1« 

rcu Typing 

Consequently, by the time data transmission from ths 
window in frame k*l start r, the camera has been comman- 
ded to mo/t (by a command Initiated after analysing 
frame k-1), and will be in the correct position. 

In a control theoretical ser.se, the plant here con- 
sists of a delay made up by vvn sampling periods (i.e. 
frame periods). In the s-transferm domain often used 
with sampled data systems, s delay of 2 periods is 
characterised by a factor s" ? , while a prediction of 
two periods means multiplication by (see figure 2). 

As is displayed on the screen, only tne relative 
position between target position and camerc. position 
is measured by the computer. To deal with occlusions, 
it is necessary to get the corrected target position 
by adding tbe camera motion. The problec solving and 
processing program (see figure 2) tries to solve this 
task in "problem" esses too, and then predicta the 
("true") target position two frames ahead in order to 
catch up the inherent delays. After processing occlu- 
sions and estimating velocities (if necessary), the 
effects of camera motion are re-inserted to provide 
an error signal for the camera servo. A digital low 
pass filter processes this error signal first. This 
filter is charged with; 

a) sx othing the email contour jumps from frame to 
frame which are due to the interlaced scanning 
techniques used in television, 

b) supplying sufficient phase reserve ahead of the 
position-error integrator in the camera control 
circuit. 

The dynamics of the closed loop are adjustable with 
the gain V (from fig. 2), so that the closed loop poles 
yield sufficient stability. As the system contains only 
one integration, an object moving with constant velo- 
city generates s stationary position error that has no 
major influence on the tracking capability. 

3.3 Treatment of croblem cases (e.g. occlusions) 

The most critical part of the control program has 
the task of checking the measured (apparent) object 
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Processing a frame implies checks for two 

>) a Jump disturbance , which occurs when suddenly one j 

OOT several contour extrema (in the following abbre- > 
viated as Ct) coincide with window edges'; in this' 
east it can be assumed that an occlusion has ente- 
red tba window; of course, for thia to work, the 
raal target aiaa never ahould exceed tha edges of 
tba window. Tha window site is continously monitored 
to insure that this assumption is true. 

b) a velocity disturbance, which occurs whan tha 
measured ioterframe change of a contour extremum 
(CD does not have tha same eign as the computed 
overall velocity. 


Taking account of tha typical structures of occlu- 
sions (treaa, poles, horixon), different algorithms 
ware developed for tha x and y directions, which, legions 
condensed form, are as follows: 


However it is clear that with using -only b matter • 

' ♦ per frame it is Impossible to adequately distinguish 

ion between occlusions an4 targets in all caeea. The system 
has particul.v* problems where an occlusion is oriented* 
** Ma ‘the direction of target motion. 'Therefore we are 4 
'-currently developing software which will be able to"^J 
(better discern occlusions by making use of all threap 
(hold transitions within the window rather than only ^ 

1 exti 





FUTURE TRENDS 
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x-direction , f 

In case of a jump disturbance or a velocity diatvr- * r 
bancs the velocity of the disturbed CE is not updated; 
if both CEa aie disturbed, pure prediction is perfor- 
med, i.e., window site and speed keep their old values. 
Moreover, in the case of a velocity disturbance in one 
CE, the disturbed CE is estimated from the latest ob- 
ject site and the other (unuisturbed ) CE. Object site 
is updated only in undisturbed cates, 

« . . . * - vu j r 

y-direction v* . «’. •• o : r *ti?n t lui 

Hare only jump disturbances are checked. When a dis- 
turbance is recognised, the corresponding CE is again 
estimated from the undisturbed one and updating of CE 
velocity is omitted. If both CKs are disturbed, predic- 
tion ia used. 

nar^i.i llrw*. 

If the growth of the y-extension taken over several 
frames exceeds a threshold, then that CE which ia the u | 
leading one with respect to object velocity ia used to 
briefly (i.e. for one frame) draw after it the other 
CE. rhen (in the next frame) it will be determined . 
whether the object site actually increased ao much or 
whether it must overcome an occlusion stationary with 
raapect to y. 

Several typical cases as they are recognised and 
solved by the program are shown in fig. 3. For example, 
it takes care that in case of a target merging hori- 1 
xontally into an occlusion the window is fixed in the y 
direction, but the window edge entering to occluaion 
triea to move 10 the end of thia occlusion and to wait 
for the target there. Independent of whether meanwhile 
the target reverses ita direction or not, the window 
1 is finally detached from the occlusion. 


Moat work in the area of computer analysis of images 
1 to data has usad "synthet ic" algorithms. That is, tha 
j tracking techniques are baaed on ideas of how we 
1 intuitively perceive that a computer could be directed 
to perform these tasks . Our thinking in developing these 
algorithms ia likewise directed by the assumption that 
I we will be using a digital computer. The methods we | 
develop may be well formulated mathematically, but they 
are, in some sense, not "natural", being baaed as they 
are on tha principles of today's serial computers. 

A major weakness of these programs is thair ina- 
bility to generalise. Juat as one cannot write a pro- 
gram which is a "little bit wrong", auch programs are 
‘ incapable of generalising to slightly different situa- 
tions. Thus, such trackers are highly dependent on 
assumed situations, for example, on the assumption 
iprthat acceleration of the target ia less than some 
threshold. 

But how is it possible to build machines wich 

°*’ r generalise? ,rltJr * ' M# 

In the late 50s and early 60s, the theory of thres- 
hold-logic devices as computational structures repre- 
senting neural models was highly toutad as the so- 
lution to all pattern recognition and cognition pro- 
* blems. These threshold logic devices seemed an similar 
to basic computer operations that tha s^in problem 
was believed to be how such elementary classification 
decisions could be combined in multilevel or hierarchi- 
cal procasaea in order to construct intelligent ana 

learning systems. 

These theories declined in popularity rapidly aa it 
became virtually impossible to find support for re- 
lated research. This decline can be attributed to 

several causes: 


Though tbe tracker initially was designed only for 


ground-air tracking with *,ood contrast, it proved in 
field experiments to show good tracking capabilities 
in ground-ground tracking with occlusion situations 
too. Thus it can be shown that by intelligent processing 
of dramatically reduced information (U values per frame) 
Deeding only 30- Ud of real time on the computer, sur- 
prisingly good rtaults could be achieved. 


1 1) The remarkable advance* in silicon technology with 
| corresponding drops in price and increases in 

capacity of conventional (von Neumann) computers. 

I 2) The observation that von Neumann machine implements 
a Turing machine and consequently can compute any 
computable function, at leaat in principle. 

3) Early overenthusiaam on the part of the neural model 
1 researchers, with corresponding claims of potential 
which far surpassed actual results. 

U) A significant underestimation of the cc^lexity of 
i the brain. 

So people became more and more interested in the use 
I of digital computers, particularly as the technological 
breakthroughs of the 60s occurred. Most of the work in 
artificial intelligence then concentrated on formulating 
problems so that they were conveniently treatable by 
digital computers. The enormous advances in technology 
prevented people from asking the critical questions 
as to whether computer development based on tbe von 
Neumann machine might turn out aa a tremendous sidetrack. 
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th* human brain la pet composed of an aggregate of 
tint 1* threshold devices, but of highly parallel. 


analog, feedback-type and even linear connections i 
r^^khat represent 4i»tr ibutiv* "memory irate*" which \ 


4tf *73 itfukv«hima,.K.*mnd Miyake, Brea ^ 

"A Sal! Organising Neural Network with A function 
Of ... Xfory". B iological Crtrrncuc. 26 . : 
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are learned during an experience and which allow - 'J 
^ the association of stored patterns from incosiplete « 
h . input keys. Reconstruct ion of patterns seems to be 

the main point of cognitive operations, and not 'Cr ^ 
Classification single ^i terns, ^ and ' 

lb) todays computers show very little similarity to the 
functioning of tbe brain. They are, as already 
mentioned, unable to get along with even slightly 
j erroneous input information unless that particular 
I case has been anticipated by tbe progrsmser ; i.e., 
tbe are unable to associate. 
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Thus, after several years of neglect, tbe concept 
'of brain models is arising again, with a better under- 
standing of neural structure and much more sophisti- 
cated models C73. Work currently underway ia also well 
founded mst beast i cal ly Cl 23. Such systems have demon- 
strated the ability to recall a sequence of previosly 
taught images when presented with a tingle eleaent from * 
tne sequence C22, 73; to recall an entire image when * 
presented with only a portion of it C123; and under 
certain constraints, to general its. • 

c Jilt v rcii£« lapur.s vill 

As yet, there no clear way to structure such an 
associative machine so that it can rscall an entire 
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image (say of a target) when presented with only a p* t<vrnpl 
portion of the image (an occluded target), which ia 
rotated and displaced in space. This property of hand- 
ling displacement is essential to the use of associa- 
tive machines as trackers. 

, . T ; * *. i -.V c i \J 1 1». 

We leel that this ares of research deserves more 
attention now that a better theory is seen to be 
evolving, as it could potentially provide solutions 
with the necessary parallelism to attain the speeds 
required for real time image analysis. 
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APPENDIX C 


USE OF THE AAC-32 IN A TRANSVERSAL FILTER APPLICATION 



2 REVIEW OP RELATED TOPICS 

% 

2.1 Digital Filter 

A digital filter is a computational process In which the sequence of 
Input numbers Is converted Into a sequence of output numbers representing 
the alteration of the data In Borne prescribed manner. Hie Input-output 

relationship can be mathematically represented as 

»* * # 

y(n) - l a.x(n-i) - l b.y(n-i) (1) 

1-0 1 1=1 1 

* 

where x(n), is the Input, y(n) is the output and a^'s and t^'s are 
constants that determine the characteristics of the filter. If all 
values of b^'s are zero, thep it Is a transversal filter. 

The realization of a digital filter may involve either hardware or 
software. In the hardware approach, it is necessary to build digital 
circuitry which can have the operations of delay, addition and nultipli- 
cation with appropriate order. In the software approach, the end product 
might be a program for a ccnputer. 

The Z-trans format ion of equation (1) is 


Y(Z) + l b.Z“‘ L Y(Z) = l a. Z _1 X(Z) 
1=1 1 i =0 1 


( 2 ) 


where Z is a complex variable. 

The transfer function H(Z) of transversal filter is defined as 

n 


l a A Z 

H , 7 s . W> . ' im ° 

H(z) xTzy — 1 


-i 


n 

1 + l b.Z 
1=1 1 


(3) 


-1 


lb examine the frequency response of the filter, it is Just simply 
to substitute Z for e*^ [33. T is the reciprocal of sampling frequency. 


2.2 Backet Brigade Device 


The bucket brigade device Is a type of charge transfer device. 

I 

These devices can move quantities of electrical charge ±n a controlled 

manner across a semiconductor substrate by applying a sequence of clock 
\ 

pulses. A fully Integrated MQS version of BED was fabricated in 1970 

[43. 

The circuit .shown in Figure 1 is known as a MOSEET EBD, because 

the activity of this circuit resembles a fire brigade of old.. Its 

basis is a chain of storage capacitors and charge-transfer circuits 

* 

acting as an analog shift register with externally variable shift rate. 


-** 



Figure 1 MQSFET EBD 


3 THE RETIOON AAC-32 


The configuration of the AAC-32 Is shown In Figure 2. There are 
two charge-transfer analog delay lines and each line has 32 Individual 
taps. Each pair of corresponding taps Is Input to a four-quadrant 
analog multiplier* The outputs of the multipliers are surnned on-chip, 

thus performing suro-of-product operations. Clocks control the shift 

* 

rate of the sampled analog signal. Each delay line also needs a 
complementary clock. Two "dead" cells offer the convenience of 
examining the sampled signal on each delay line. This particular 
device was evaluated by ^yder [4], Some useful conclusions are 
sunmarlzed below. 

Clock rate . *The AAC-32 will function well with clock rates as 
high as lOOKHz, 

Four quadrant multiplication test. In the four quadrant multi- 
plication test, it was shown that the device had a tendency to clip* in 
the third quadrant. 















4 EESXON AND IWIEMEOTATTON 


4.1 Deslfen Considerations 

4.1.1’ ’ Clock frequency 

V The clock frequency of AAC-32 is essentially the sanpllng frequency 
for the Input signal* For a baseband Input signal In filtering, lower 
sanpllng frequency has the better performance. This conclusion comes’ 
from the result of ccnputer simulation. In Figure 3» transfer functions 
of different sanpllng rates at 60, 80 and lOOKHz were simulated and 
plotted. The transfer function for a sanpllng rate of 60 KHz has the 
sharpest cutoff at stopband. It seems good to have a possibly low 
sanpllng frequency In this filter design. However, sanpllng frequency 
was set at 100 KHi In order to have minimum idle time (A second order 
distortion was observed to be a function of the time during which the 
clock was Idle). 

4.1.2’ ' Tap weight 3 

The transversal filter was designed to be a lOKHz low-pass filter. 
An 8-bit PROM was programmed to provide weighting coefficients. Com- 
puter simulations was completed to examine the effect of quantizing tap 
weights for this filter design. Since the Fourier series method [53 was 
used In determining the weighting coefficients, only an odd number of 
coefficients was required for a finite lirpulse response filter design. 
Thus the 32nd tap was set to have value zero. An examination was also 
made to pick up the first 32 weighting coefficients from the 33-coeffi- 
cient filter design under the same specifications . A Hamming window [5] 
has been used In evaluating weighting coefficients in order to overcome 
the Gibbs’ phenomenon [53. 


lOOKHz 
- 80KHz 
. 60KHz 






1 s -0 10. IS. 20. 25. 30. 35. <10. <15. 50. 55. 60 

FREQ. (KHZ) 

Figure 3. Freqjency response of a lov-pass filter v.lth 
different sarpllnr rate 6C, 80, lOQiOiz 



The results of simulations are summarized. In the followings 

The quantization of weighting coefficients has a significant 
effect on the stopband attenuation. Were Is about 13 dB difference 
between the maximum ripple In the stopband (Figure 4). Also the cal- 
culated response of the filter having exactly computed weighting 
coefficient has a sharper cutoff. These conclusions hold for 31- and 
32-coefficient filters In this design. Figure 4 Is the simulation of a 
31-coefficlent filter and Figure 5» for a 32-coefficient filter. The 
dashed line represents the frequency response of the filter with 
weighting coefficients quantized to eight bits, ihe solid line gives 
the frequency response with weighting coefficients canputed 32 bits 
accuracy. Comparing the frequency responses of 31 quantized coefficients’ 
filter with that of 32 quantized coefficients, 31 coefficients' filter 
■has slightly better performance. Ihe 31-coefficient filter wlll.be used 
in the Implementation of a practical filter. Figure 6 shows the 
frequency responses of these two filters. 

4.1.3 Charge transfer inefficiency 

The charge transfer Inefficiency [63 is defined to be the fraction 
of charge left In one stage when charge is transferred to the next. 

The overall charge Inefficiency is proportional to the number of stages 
If the fraction is constant and Independent of signal charge. Typically, 
a device with less than 10000 stages is acceptable [1]. Transfer 
Inefficiency did not appear to affect the performance of the filter. 
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Figure Jj. 31-Coefficlent low-pass filter 


J 








AMP. (DB) 

-HO. -120. -100. -80.0 -60.0 -H0.0 -20.0 O 20.0 



FREQ. (KHZ) 


Figure 5. 32-Coerflcler.l low-past Tllter 
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Figure C . 8— fcii transversal filter 







4,2 Implementation 
4.2.1 Loading weighting c oefficients 

An AAC-32 can ccnpute a sum of products of two sanpled analog 
sisals. Hie output Is 

31 

y - Z a ± x ± (4) 

1-0 1 1 

*' > 4 

where y Is the analog output and the a^'B and x^'s are the samples of 

the analog Input signals controlled by the external clock. The trans- 
versal filter has the following input -output relationship: 

y(n) - l a.xOvl) . (5) 

,1=0 

Here y(n) and x(n) are samples of the input and output signals 

» t 

respectively. Hie a^'s are weighting coefficients which determine the 
characteristics of the filter, 'lb use the AAC-32 for the computation 
of Equation (5), we must keep the a i ’s constant while the x^s are 
being clocked through the delay line. It would be desirable to sinply 
load the coefficients into the CH), and leave them alone, while passing 
the data through the filter. Unfortunately, the coefficients are stared 
in the form of charge on capacitors, and will decay if not refreshed 
periodically. A method of refreshing which 1 b general to this class of 
devices la shown in Figure 7» with timing diagrams in Figure 8. The 
general concept is one of time nultiplexing. The coefficients of 
BED are refreshed while the other 1 b conputing, and the roles are 
reversed every 32 clock cycles. All a^'B are clocked into one delay 
line of one of these devices and held 320 ysec; -that cycle then repeats. 
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Figure 8. Timing diagrams 





(At a sampling frequency of lOOKHz, 32 delays results In 320 ysec 
delay.) » 

14.2.2 Configuration 

Ibe overall configuration of this Implementation is illustrated 
in Figure 9. It can be divided into three main portions. These are 
control, data processing and data acquisition. Each part will be 
discussed separately. 

14.2.3 Control 

AAC-32 requires a two-phase c orrplement ary square-wave clock drive. 

A free-running variable frequency clock provides the time base. Two- 
phase canpleraentary square-wave clock and the clocking waveforms which 
control the loading of coefficients in Figure 8 are Implemented by using 
combinational logic as shown in Figure 10. 

f 

* Data processing 

The input Bigpal with 1 volt swing passes through a level shifter 
to have 'zero' at 5.6v. Since the multipliers have biasing problems 
in the third quadrant, we are farced to have all signals higher than 
5.6v after passing the level shifter. Ihus we are actually computing 

31 31 

y(n) « i a.x(n-i)- + £ a.c • (6) 

i-0. 1 i=0 1 

where c is the input DC offset with respect to 5*6v« 

Two AAC-32 's were used to process input si gna ls. Since we need 
320ysec to refresh weighting coefficients, only 320nsec of processed 
data of every 6^0psec output is valid for each AAC-32. The output cir- 
cuit is a dif ferent ial amplifier which is a current-to-voltage converter 
(Figure 11). 



Figure 9. The configuration of overall system 
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H.2.5 Data acquisition 

An analog switch is used to select the valid data from the outputs 

of the current-to-voltage converters. Every 320ysec, the analog switch 

passes one output and cuts the other. Following the analog switch, a 
V 

sanple and hold circuit is used to hold valid data, because only when 
both of the driving clocks of the AAC-32 are high is the data valid 
(refer to Figure 7). The sample pulse is delayed about 3vsec ,to allow 
the data to settle. A simple low-pass filter can be connected to the 

output of the sample and hold circuit to eliminate high frequency 

* 

component due to clocking. 

** *J.3 Results 

Figure 12 sh6ws the frequency response of a low-pass transversal 
filter compared to the calculated filter response fran DC to l^KHz. 
These two sets of data are quite consistent . 
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Figure 12. Frequency -response plots for 8-bit accuracy 

31 -coefficient low-pass filter. Bores represent 
experimental values for the frequency response 




5 SUMMARY AND CONCLUSIONS 


The overall operation of a transversal filter can be described as 
a cross-correlation of the sanpled input signal and weighting coeffi- 
cients. If the weighting coefficients can be treated as another sampled 
Input signal, then a transversal filter is a remarkably versatile 
instrument for processing signals with different characteristics. A 

it 

different response can be obtained by simply changing the signal source 
which generates the weighting coefficients. 

This thesis discusses* the implementation of a progranmable 32-tap 
charge-transfer transversal filter. The AAC-32 was used as a primary 
device to implement a low-pass filter with tap weights controlled exter- 
nally. An 8-bit FkjM was programmed to store the tap weights. It was 
used to generate the coefficients for the designed filter, and the 
coefficients may be changed simply by replacing the PROM. The amplitude 

response of this filter is consistent with theoretical prediction. 

* 

Since both the 'weights and signal are stored in charge transfer 
devices, the weights are volatile and must be refreshed periodically. 
Additional circuitry is required to accorrplish this refreshing. 

The complexity and cost in design and implementation of this 
transversal filter can be reduced significantly if one can have an on- 
chip static binary shift register or Programmable Read-Only Memory to 
hold the weights. We understand that semiconductor manufacturers are 
now developing such a product. 
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PROGRAM LISTINGS 
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TEST: 


INI: 
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•TITLE RAMTSB. MAC VERSION 8 7 


. MCALL . PRINT , .EXIT 


CSR1=167770 
CSR2=1 67760 
0UT2*=1 6 < 7762 
IN2=167764 
OUT1 =167772 


J DEFINE LOG. NAME FOR CONTROL 
! STATUS REGISTERS 
iDATA INPUT ASSIGNMENT 

jdata dutput assignment 

J OUTPUT CARD 1 


NOV 

“• * SP 

INITIALIZE SP 




BIC 

SJl40»5>ttCSRl- 

J CLEAR AFFECTED 

BITS OF CSR1 

» 

BIC 

»140,5>«CSR2 

J CLEAR AFFECTED 

BITS OF CSR2 


BIC 

*:i 77777, 9«0UT2 

J RESET THE DATA 

OUTPUT 



CLR 

R1 

5 INITIALIZE TD ZERD ALL DATA 


BIS 

«U,R1 

5SET TO 1 THE 0 

BIT OF 

THE DATA 

OUTPUT 

JSR 

* PC, TEST 

5 INITIALIZE FDR 

OUTPUT 

BITS 


BIS 

«2,R1 

?SET TD 1 THE 1 

BIT OF 

THE DATA 

OUTPUT 

JSR 

PC, TEST 

J INITIALIZE FOR 

OUTPUT 

BITS 


BIS 

«4,R1 

JSET TO 1 THE 2 

BIT DF 

THE DATA 

DUTPUT 

JSR 

PC, TEST 

5 INITIALIZE FDR 

DUTPUT 

BITS 


BIS 

::10,R1 

JSET TO 1 THE 3 

BIT OF 

THE DATA 

OUTPUT 

JSR 

PC, TEST 4 

UNITIALIZE FOR 

OUTPUT 

BITS 


BIS 

-«20,R1 

JSET TO 1 THE 4 

BIT DF 

THE DATA 

OUTPUT 

JSR 

PC, TEST 

UNITIALIZE FOR 

DUTPUT 

BITS 


BIS 

«:40,R1 

• J SET TO 1 THE 5 

BIT OF 

THE DATA 

OUTPUT 

JSR 

*PC, TEST 

*-* INITIALIZE FDR 

OUTPUT 

BITS 

* 

BIS 

•«100,R1 

JSET TO. J THE 6 

BIT OF 

THE DATA 

DUTPUT 

JSR 

PC, TEST 

UNITIALIZE FOR 

.OUTPUT 

BITS 


BIS 

4*200, R1 

•JSET TO 1 THE 7 

BIT DF 

THE BATA 

OUTPUT 

JSR 

"PC, TEST 

UNITIALIZE FOR 

DUTPUT 



CLR 

~R1 

J CLEAN THE DATA 

OUTPUT 



• EXIT 


JEND OF PROGRAM 





MOV 

-Rl,«ttDUT2 

J OUTPUT THE DATA SET 

MOV 

«0UT1,R4 

J STORE DATA OUTPUT ASSIGN INTO R4 

BIC 

»1 00000, ®R4 

J ENABLE THE BUS DRIVERS 

CLR 

R3 

J INITIALIZE TD ZERD EIGHTS COUNTER 

CLR 

R2 

J INITIALIZE TD 'ZERO UNITS COUNTER 

BIC 

s*340?,S»R4 

JCLFAR ADDRESS BITS 

JSR 

PC, CHECK 

JCRlL check SUBROUTINE 

INC 

9P.4 

J INCREMENT IN 1 THE LOU ADDRESS BITS 

INC 

R2 

J INCREMENT THE UNIT COUNTER 

CMP 

R2,«7 

JTEST FDR COUNT TO 7 

BLE 

INI 

J BRANCH TD ROUTINE 

CMP 

S7,R3 

J CHECKS TOP OF HIGH ADDRESS BITS 

BEQ 

NEXT 

J EXIT THE PROGRAM IF COUNT IS FINISHED 

BIC 

Si7,S»R4 

J CLEAR THE LOW BIT ADDRESS 

ADD 

«»400,9R4 

? INCREMENT 1 UNIT OF HIGH BIT ADDRESS 

INC 

R3 

J INCREMENT THE EIGHTS' COUNTER 

CLR 

R2 

J INITIALIZE TO ZERD UNITS COUNTER 

JMP 

INI 

J INITIALIZE COUNT WITH LOU ADDRESS 


NEXT: 

.PRINT 

«MES£ ‘ “ 

? INDICATE END DF TESTING DF DNE BIT 


CLR 

R1 

5 RESET VALUES OF THE OUTPUT BITS 

• 

RTS •* * 

PC 

f RETURN TO ROUTINE TESTING THE BITS 

CHECK: 

B1C 

«50000> 9R4 

> WRITE TD THE RAM 

, # 

BIS 

vl 0000> 9R4 

? READ THE RAM 


CMPB 

R1»?::IN£ 

5 COMPARE THE INPUT AND ECHOED BITS 


BEQ 

RET 

•JIF DIFFERENT WRITE MESSAGE 

„ ,»• rr - 

.PRINT 

wMESSAG 



BIC 

«1 77777 f ?::DUT£ 

; RESET THE OUTPUT DATA 


.EXIT 



RET: 

• 

RTS 

PC 

9 RETURN VIA PC 

MESS AG: 

. RSCIZ 

/'APPARENT ERRDR 

IN RAM/' 

MESS: 

. RSCIZ 

✓FINISHED DNE BIT/' 


-£ND START fMAKE PROGRAM SELF STARTING 




I 


RAM TEST PROGRAM 




I' . MCALL . PRINT* .EXIT 

;this is a short version of ram testing program. 

I 5 IT PASS ALL 1 \S TO RAM AND CHECK THE RAM OUTPUTS. 

J J THEN IT PASS ALL 0'S TO RAM AND CHECK THE RAM OUTPUTS. 


; initialization; 

.PSECT 


f 

,1 

OPl=16777£ 


0P£=16776£ 


IP£= 167764 

| START: 

MOV ' 

*SP 


BIC 

«: 1 4 0 * 9:: 1 6777 0 


BIC 

«140*9::16776G 

1 

MOV 

*>50000, R3 

1 

MOV 

*>50000, R4 


CLP. 

9*>DP1 


CLR 

3:>DP£ 

BEGIN: 

JMP 

TEST 

CK: 

CMP 

*>54000? R4 


BNE 

BEGIN * 

RETURN: 

.PRINT 

.EXIT 

*>TXT£ 

TEST: 

CLR 

9*>0P1 

«a» 

CLR 

3**0P£ 


MOV 

R3,9*>0P1 


MOV 

*>377,9*>OP£ 


BIC • 

*>40000, 9*>0P1 


BIC 

.*>1 0000* 9*>OPl 


BIS 

*>10000,9<>DP1 


BIS 

*>40000,9**OP1 

1 

MOV 

R4,9*>0P1 

1 

BIC 

*>40000, 9*>DP1 


CMPB 

*>377,5>*>IP£ 


BEG 

FIRST 

1 

.PRINT 

*>TXT 

FIRST: 

CLR 

9*>DP1 


CLR 

•><>OP£ 


MOV 

R3,9*>0P1 

- 

MOV 

s:0» 9*>0P£ 


BIC 

*>40000, 9*>0P1 


BIC 

*>1 0000, 9*>DP1 


BIS 

*>10000»3*>OP1 

I 

BIS 

*>40000, 3*>0P1 

1 

MOV 

R4,5»*>0P1 


BIC 

<>40000, 3*>0P1 

1 

CMPB 

*>0,9*>IP£ 

| 

BEG 

SECOND 


.PRINT 

<>TXT1 


;load stack pointer 

5 SET CSR OF PORT 1 
JSET CSR OF PDRT £ 

JSET CONTROL FORMAT 
J SET CONTROL FORMAT 
J CLEAR OUTPUT PORT 1 
> CLEAR OUTPUT PORT £ 

J BEGIN TESTING 

J COMPARE THE REQUIRED NO. OF TEST 
J BRANCH IF NOT FINISH 
J INDICATE FINISH TEST 

;exit td monitor 

J CLEAR OUTPUT PORT 1 
; CLEAR OUTPUT PORT £ 

JPUT CONTROL FDRMAT AT OUTPUT PORT 1 
5 LOAD DATA TO OUTPUT PORT £ 

J ENABLE RAM 

5PUT RAM AT WRITE MODE 
J DISABLE RAM AT WRITE MODE 
5 DISABLE RAM 

JPUT CONTROL FORMAT AT OUTPUT PORT 1 
i ENABLE RAM 

I COMPARE THE ECHOED DATA 
5 BRANCH IF NO ERROR 
5 OTHERWISE INDICATE ERROR TYPE 1 
J CLEAR OUTPUT PORT 1 
J CLEAR OUTPUT PORT £ 

JPUT CONTROL FORMAT AT OUTPUT PDRT 1 
J LOAD DATA TD PUTPUT PORT £ 

J ENABLE RAM 

JPUT RAM AT WRITE MODE 
J DISABLE RAM AT WRITE MODE 
J DISABLE RAM 

JPUT CONTROL FORMAT AT OUTPUT PORT 1 
J ENABLE RAM 

J COMPARE THE ECHOED DATA 
J BRANCH IF ND ERRDR 
J OTHERWISE INDICATE ERROR TYPE £ 




SECOND! 

CMPB 

«?>R3 



; check: lowest 3 address bits 


BNE 

GO 



t BRANCH IF NOT FULL 


JNP 

THIRD 



i OTHERWISE JUMP TO THIRD 

GO: 

INC 

R3 



5 NEXT TEST LOCATION 


INC 

R4 



• 

r 


JNP 

CK 



JBACK TO THE MAIN PROGRAM 

THIRD: 

BIC v 

«7>R3 



? RESET LOWEST 3 ADDRESS BITS 


BIC 

it? r R4 



m 


ADD 

«400»R3 

» 


; INCREMENT HIGER 3 ADDRESS BITS 


ADD 

«400»R4 



m 

t 


JNP 

CK 



;back to the main program 

TXT: 

. ASCIZ 

✓ERROR: 

TYPE 

is 

CHECK RAM/ 

TXT1 : 

.ASCIZ 

✓ERROR: 

TYPE 

£: 

CHECK RAM/ 

TXT2: 

.ASCIZ 

✓FINISH 

TESTING/ 


.END 

START 





I 


ZEROA PROGRAM 


THIS IS A PRQGRAM TO ADJUST THE ZERO OF A SIDE OF BED. 

S/H STROBE KEEPS HIGH ALL THE TIME. 

THE SIGNAL INPUTS TO B SIDE OF BBD HAS SAWTOOTHED PATTERN. 

THE PROGRAM CAN BE TERMINATED VIA CONSOLE BY USE THE COMMAND CONTROL C. 

INITIALIZATION: 

** » 

. PSECT 
□Pl=167772 
□P2= 167762 


LOADING RAM WITH 'O' AT LOCATION 0: 


START: 

MOV 

. > SP 

* LOAD STACK POINTER 


BIC 

«140*3« 167770 

5SET CSR OF PORT 1 


BIC 

«! 40, 9tt 167760 

*SET CSR OF 1 PORT £ 


MOV 

«150000>R2, 

* STORE THE INITIAL CONTROL FORMAT FOR LODING 


MDV 

R£*Su:QPl 

5 CONTROL FORMAT FOR LOADING RAM 


MDV 

«376* 5>::OP£ 

JLOAD RAM WITH 'O' 


BIC 

«4 0000>JUfDPl 

* ENABLE RAM 

• 

BIC 

::10000*J*::DP1 

* INPUT DATA TO RAM 


BIS 

«10000,J>::OP1 

■ 

** 

• 

BIS 

«40000* 5>~QP1 

J DISABLE RAM 

J INPUT 
• 

DATA 

» 

TO bbd AND ADJUST 

THE ZERO OUTPUT ON R/I BOARD: 

LOOP: 

MOV 

“130330* R£ 

J INITIAL CONTROL WORD TO LDAD DATA TO BBD 


MOV 

«1 77000* 5>::DP£ 

J LOAD DATA ' 0' TO D/A CONVERTER 


JSR 

PC* DATA 

J OUTPUT DATA FROM RAM TO D/A CONVERTER 
* AND LOAD DATA TO BBD 


MOV 

«1400*»-DP£ 

JLOAD DATA '+l/£' TO D/A CONVERTER 


JSR 

PC* DATA 

» 


MDV 

::i77400*i<^OP£ 

JLOAD DATA '+1' TO D/A CONVERTER 


JSR 

PC* DATA 

• 

* 


MOV 

::1400» S::0P£ 

JLOAD DATA '■*■!/£' TO D/A CONVERTER 


JSR 

PC* DATA 

• 

* 


MOV 

«1 77000 *i>«DP£ 

JLOAD DATA ' 0' TO D/A CONVERTER 


JSR 

PC* DATA 

* 

* 


MOV 

::1000*9::DP£ 

JLOAD DATA '-l/£' TO D/A CONVERTER 


JSR 

PC* DATA 

• 

* 


MOV 

«0*J>::DP£ 

JLOAD DATA '-1' TO D/A CONVERTER 


JSR 

PC* DATA 

• 

* 


MOV 

::1000*9«DP£ 

JLOAD DATA '-1/E' TO D/A CONVERTER 


JSR 

PC* DATA 

J 


BR 

LOOP 

J CONTINUE THE PROCESS 


? SUBROUTINE TO LORD DATA TO BBD? 


m 

DATA* BIS 
MOV 
BIS 
BIC 
BIC 
MOV 
BIS 
BIG 
RTS 


**10030>R2 
R2*S>«DP1 
«40»9«OP1 
V «40»5>«OP1 
«30>R2 
R2» JPwOPl 
«40>5>«OP1 
«40»#>«OP1 
PC 


5 PULL UP THE CLOCK 1A fc IB 

■ 

?PULL UP THE MASTER CLOCK 

5 PULL DOWN THE MASTER CLOCK 

? PULL DOWN THE CLOCK lfi 8. IB 
• 

5 PULL UP THE MASTER CLOCK 
5PULL DOWN THE MASTER CLOCK 
? RETURN VIA PROGRAM COUNTER 


* 


END ' START 


ZEROB PROGRAM 



THIS IS A PROGRAM TO ADJUST THE ZERO OF B SIDE OF BBD. 

S/H STROBE KEfePS HIGH ALL THE TIME. 

THE SIGNAL INPUTS TO A SIDE OF BBD HAS SAWTOOTHED PATTERN. 

THE PROGRAM CAN BE TERMINATED VIA CONSOLE BY USE THE COMMAND CONTROL C. 


INITIALIZATION: 

.PSECT 
DPl=i 67772 
0P2=1 67762 

LOADING RAM: 


START: 


MOV 

«. y SP 

J LOAD STACK POINTER 

BIC 

# 1 4 0»9«* 167770 

J SEJ CSR OF PORT 1 

BIC 

»140r 167760 

J SET CSR OF PORT £ 

MOV 

•>150000 * R£» 

J CONTROL FORMAT FDR LOADING 1 

MOV 

R2*S>«0P! 

m 

9 

MOV 

::376»5>«OP£ 

J LOAD RAM LOCATION ZERO WITH 

JSR 

PC » LOAD 

• 

9 

INC 

R2 

JNEXT LOCATION 

MOV 

R2»5>«0P1 

* 

f 

MDV 

n ■«374»5'«0P2 

J LOAD RAM WITH '+X/£' 

JSR 

PC» LOAD 

m 

y 

INC 

R2 

JNEXT LOCATION 

MOV 

*R2>9«0P1 

J 

MOV 

kO,9«OP2 

J LOAD RAM WITH '•+! ' 

JSR 

PC » LOAD 

m 

y 

INC 

R2 

JNEXT LOCATION 

MOV 

R2»9«0P1 

• 

y 

MDV 

s>374> 5>s>QP£ 

JLOAD RAM WITH '+1/2' 

JSR 

PC » LOAD 

• 

INC 

R£ 

JNEXT LOCATION 

MDV 

R2,S<::0P1 

• 

y 

MOV 

«376*5«0P£ 

JLOAD RAM WITH ' 0' 

JSR 

PC » LOAD 

m 

y 

INC 

R£ 

JNEXT LOCATION 

MOV 

R2»9«0P1 

- m 

y 

MOV 

t>375* 5>s:OP£ 

JLOAD RAM WITH "-l/£ / 

JSR 

PC » LOAD 

• 

9 

INC 

R2 

JNEXT LOCATION 

MDV 

R£» S»“0P1 

■ . 

y 

MOV 

«377>S>«0P£ 

JLOAD RAM WITH '-1' 

JSR 

PC» LOAD 

5 

INC 

R£ 

J NEXT LOCAT I DN 

MDV 

T2»*>s*DP 1 

■ 

y 

MOV 

»375»0«0P£ 

JLOAD RAM WITH '-1/2' 

JSR 

PC* LOAD 

• 

9 


<mw «• '•i 


INPUT DATA TD BBD RND ADJUST THE ZERO OUTPUT ON fi/I BDARD: 


LOOPS 


LORD: 


MOV 

«150300*S>«DP1 

* DISABLE RRM 

MOV 

#400* S>((DP£ 

5 LORD <X-U>'S B/R WITH 'O' 

MOV 

:: 130330* RE 

UNITIALIZE CONTROL WORD TD LORD DRTR 

BIS 

«10030»R2 

IPULL UP THE CLOCK 1R fc IB 

MOV 

R2*S>:*0P1 

■ 

f 

BIS 

«40*?::OP1 

5 PULL UP THE MRSTER CLOCK 

BIC 

«40*?«DP1 

IPULL DOWN THE MRSTER CLOCK 

BIC * 

c30*R2 

IPULL DDWN THE CLOCK 4fi t, IB 

MOV 

R2»»:sOPl 

• 

f 

BIS 

«40*5»::DPJ 

IPULL UP THE MRSTER CLOCK 

BIC 

w40*9s:OPJ 

IPULL DOWN THE MRSTER CLOCK 

INC 

R2 

UNCREMENT RRM ADDRESS 

CMPB 

«310»R2 

J CHECK LOWEST 3 ADDRESS BITS 

BNE 

LODP 

I CONTINUE TESTING 

BIC 

:: 1 0 > R2 

JELSE RESET LOWEST 3 ADDRESS BITS 

BP 

LOOP 

* CONTINUE TESTING 

TINE TD 

LORD RRM: 


BIC 

«40000*S , «OP1 

I ENABLE RRM 

■BIC 

«10000*5>:sDPl 

UNPUT DRTR TO RAM 

BIS 

•til OOGO*5::OP1 

• 

•* 

BIS 

s»40000*9«:OP1 

v DISABLE RRM 

RTS 

PC 

? RETURN VIA PRDGRRM COUNTER 

.END 

STRRT 

• 


